Work machine control system, and work machine control method

Abstract

To adjust, with a small number of trials, control parameters to control excavation works.SOLUTION: A work machine control system comprises a work machine that has a bucket at its front end and performs excavation works with the bucket, and a controller that adjusts control parameters to control the excavation works. The controller, being a function of the control parameters, adjusts the control parameters with use of an evaluation function that involves weight of the load taken with the bucket during an excavation work and cost taken by the excavation work. The controller, in the process of adjusting the control parameters, takes into considerations both an estimated average value of the evaluation function and uncertainty in the estimation of the evaluation function, and then determines control parameters to be evaluated next.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present disclosure relates to a control system for a work machine and a control method for a work machine. [Background technology]

[0002] Japanese Patent Publication No. 2023-10363 (Patent Document 1) discloses a control system that calculates a control quantity for driving a controlled object, which is a work machine, based on an operation target of the controlled object, an operation evaluation index, and an operation model, and updates the operation model based on the state quantity of the controlled object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-10363 Summary of the Invention [Problem to be solved by the invention]

[0004] Among the tasks performed by work machines, excavation work, which involves digging a target for excavation, requires advanced skills to adapt to various on-site conditions. Furthermore, excavation work consumes a lot of fuel. To ensure high productivity regardless of the operator's skill, automatic control of excavation work is being considered. There is a demand for the control parameters used to control excavation work to be appropriately adjusted with a small number of trials.

[0005] The present disclosure proposes a technique for adjusting control parameters for controlling an excavation operation with a small number of trials. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, there is provided a control system for a work machine, the control system including a work machine having a bucket at its tip and performing excavation work with the bucket, and a controller that adjusts control parameters for controlling the excavation work. The controller adjusts the control parameters using an evaluation function that is a function of the control parameters and that includes the weight of a load to be loaded into the bucket during the excavation work and the cost of the excavation work. In the process of adjusting the control parameters, the controller determines the next control parameter to be evaluated, taking into account both the predicted average value of the evaluation function and the prediction uncertainty of the evaluation function.

[0007] A method for controlling a work machine according to one aspect of the present disclosure includes the following steps. The first step is to calculate an evaluation function, which is a function of control parameters for controlling excavation work performed by a bucket at the tip of the work machine, and which includes the weight of a load to be loaded onto the bucket during the excavation work and the cost of the excavation work. The second step is to adjust the control parameters using the evaluation function. In the process of adjusting the control parameters, the next control parameter to be evaluated is determined taking into consideration both the predicted average value of the evaluation function and the prediction uncertainty of the evaluation function. [Effects of the Invention]

[0008] According to the present disclosure, control parameters for controlling an excavation operation can be adjusted with a small number of trials. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an external view of a hydraulic excavator. [Figure 2] FIG. 2 is a side view of the hydraulic excavator showing the configuration of sensors attached to the work equipment. [Figure 3] 1 is a block diagram showing a schematic configuration of a control system for a work machine. [Figure 4] FIG. 2 is a schematic diagram of a work machine for explaining moment balance. [Figure 5] 10A and 10B are diagrams illustrating the operation of a work machine by automatic excavation. [Figure 6] 10 is a flowchart showing the flow of a process for optimizing a control parameter. [Figure 7] FIG. 1 is a diagram illustrating a first example of a Gaussian process model. [Figure 8] FIG. 10 is a diagram illustrating a second example of a Gaussian process model. [Figure 9] FIG. 10 is a diagram showing a convergence curve of an evaluation function. [Figure 10] FIG. 10 is a diagram showing the relationship between the amount of soil and the amount of work for each weighting coefficient. [Figure 11] FIG. 10 is a diagram showing an optimal solution of control parameters for each weighting coefficient. [Figure 12] 10 is a flowchart showing the flow of a process for optimizing control parameters when Safe Opt is introduced. [Figure 13] FIG. 10 is a diagram illustrating a third example of a Gaussian process model. [Figure 14] FIG. 10 is a diagram illustrating a fourth example of a Gaussian process model. [Figure 15] FIG. 10 is a diagram illustrating a fifth example of a Gaussian process model. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the embodiments will be described with reference to the drawings. In the following description, the same parts and components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated. In the drawings, configurations may be omitted or simplified for the sake of convenience. It is also intended from the beginning that any configurations may be extracted from the embodiments and arbitrarily combined.

[0011] <Work machine configuration> In the embodiment, the work machine will be described by taking as an example a hydraulic excavator 100. Fig. 1 is an external view of a hydraulic excavator 100 as an example of a work machine.

[0012] As shown in Fig. 1, the hydraulic excavator 100 has a main body 1 and a hydraulically operated work machine 2. The main body 1 has a revolving body 3 and a running body 5. The running body 5 has a pair of tracks 5Cr and a traveling motor 5M. The traveling motor 5M is provided as a drive source for the running body 5. The traveling motor 5M is a hydraulic motor that is hydraulically operated.

[0013] The running body 5, more specifically the crawler belt 5Cr, is in contact with the ground during operation of the hydraulic excavator 100. The running body 5 can travel on the ground by rotation of the crawler belt 5Cr.

[0014] The rotating unit 3 is disposed on the running unit 5 and is supported by the running unit 5. The rotating unit 3 is movable relative to the running unit 5. The rotating unit 3 is mounted on the running unit 5 so as to be rotatable relative to the running unit 5 around a rotation axis RX. The rotating unit 3 is attached to the running unit 5 via a rotating circle portion, so that the rotating unit 3 is rotatable relative to the running unit 5.

[0015] The rotating unit 3 has a cab 4. An occupant (operator) of the hydraulic excavator 100 sits in this cab 4 to operate the hydraulic excavator 100. A driver's seat 4S is provided in the cab 4 where the operator sits. From within the cab 4, the operator can operate the work implement 2, can operate the rotating unit 3 relative to the traveling unit 5, and can also operate the traveling of the hydraulic excavator 100 using the traveling unit 5. In the present disclosure, the hydraulic excavator 100 is operated from within the cab 4, but it may also be remotely controlled by wireless from a location away from the hydraulic excavator 100.

[0016] In the embodiment, the positional relationship of each part in the rotating body 3 of the hydraulic excavator 100 will be described with reference to an operator seated in the driver's seat 4S in the cab 4. The front-to-rear direction refers to the front-to-rear direction of the operator seated in the driver's seat 4S. The direction facing the operator seated in the driver's seat 4S is the forward direction, and the direction behind the operator seated in the driver's seat 4S is the rearward direction. The left-to-right direction refers to the left-to-right direction of the operator seated in the driver's seat 4S. The right and left sides of the operator seated in the driver's seat 4S when facing directly ahead are the right and left directions, respectively. The up-to-down direction refers to the up-to-down direction of the operator seated in the driver's seat 4S. The side near the feet of the operator seated in the driver's seat 4S is the down side, and the side above the head is the up side.

[0017] In the front-to-back direction, the side where the work implement 2 protrudes from the swivel body 3 is the front direction, and the opposite direction is the rear direction. Looking forward, the right and left sides in the left-right direction are the right and left directions, respectively. In the up-down direction, the side with the ground is the bottom side, and the side with the sky is the top side.

[0018] The rotating body 3 has an engine room 9 that houses an engine, and a counterweight provided at the rear of the rotating body 3. The engine room 9 contains an engine that generates driving force, a hydraulic pump that receives the driving force generated by the engine and supplies hydraulic oil to a hydraulic actuator, and other components.

[0019] The work implement 2 is mounted on the rotating structure 3 and supported by the rotating structure 3. The work implement 2 has a boom 6, an arm 7, and a bucket 8. The boom 6 is rotatably connected to the rotating structure 3. The arm 7 is rotatably connected to the boom 6. The bucket 8 is rotatably connected to the arm 7. The bucket 8 is disposed at the tip of the work implement 2. The tip of the bucket 8 is called the cutting edge 8a. The bottom surface 8b is part of the outer surface of the bucket 8. The bottom surface 8b is formed as a flat surface.

[0020] The base end of the boom 6 is connected to the rotating unit 3 via a boom foot pin 13. The boom 6 is rotatable relative to the rotating unit 3 around the boom foot pin 13. The base end of the arm 7 is connected to the tip of the boom 6 via an arm connecting pin 14. The arm 7 is rotatable relative to the boom 6 around the arm connecting pin 14. The bucket 8 is connected to the tip of the arm 7 via a bucket connecting pin 15. The bucket 8 is rotatable relative to the arm 7 around the bucket connecting pin 15. The boom foot pin 13, arm connecting pin 14, and bucket connecting pin 15 extend substantially in the left-right direction.

[0021] The work implement 2 has a boom cylinder 10, an arm cylinder 11, and a bucket cylinder 12. The boom cylinder 10 drives the boom 6. The arm cylinder 11 drives the arm 7. The bucket cylinder 12 drives the bucket 8. The boom cylinder 10, the arm cylinder 11, and the bucket cylinder 12 are each a hydraulic cylinder driven by hydraulic oil. The boom cylinder 10, the arm cylinder 11, and the bucket cylinder 12 constitute a work implement actuator that drives the work implement 2.

[0022] FIG. 2 is a side view of the hydraulic excavator 100 showing the configuration of sensors attached to the work implement 2.

[0023] A pressure sensor 10A is attached to the head side of the boom cylinder 10. The pressure sensor 10A can detect the pressure (head pressure) of the hydraulic oil in the cylinder head-side oil chamber of the boom cylinder 10. A pressure sensor 10B is attached to the bottom side of the boom cylinder 10. The pressure sensor 10B can detect the pressure (bottom pressure) of the hydraulic oil in the cylinder bottom-side oil chamber of the boom cylinder 10.

[0024] A pressure sensor 11A is attached to the head side of the arm cylinder 11. The pressure sensor 11A can detect the head pressure of the arm cylinder 11. A pressure sensor 11B is attached to the bottom side of the arm cylinder 11. The pressure sensor 11B can detect the bottom pressure of the arm cylinder 11.

[0025] A pressure sensor 12A is attached to the head side of the bucket cylinder 12. The pressure sensor 12A can detect the head pressure of the bucket cylinder 12. A pressure sensor 12B is attached to the bottom side of the bucket cylinder 12. The pressure sensor 12B can detect the bottom pressure of the bucket cylinder 12.

[0026] A stroke sensor 18A is attached to the boom cylinder 10. A stroke sensor 18B is attached to the arm cylinder 11. A stroke sensor 18C is attached to the bucket cylinder 12. The stroke sensors 18A, 18B, and 18C detect the displacement of the cylinder rod in each cylinder.

[0027] 2, in a side view, the angle formed by a line passing through the boom foot pin 13 and the arm connecting pin 14 and a line passing through the boom foot pin 13 and parallel to the ground G is defined as boom angle θ1. The boom angle θ1 is calculated based on the sensor output of the stroke sensor 18A in the boom cylinder 10.

[0028] In a side view, the angle formed by a line passing through the boom foot pin 13 and the arm connecting pin 14 and a line passing through the arm connecting pin 14 and the bucket connecting pin 15 is defined as the arm angle θ2. The arm angle θ2 is calculated based on the sensor output of the stroke sensor 18B in the arm cylinder 11.

[0029] In a side view, the angle formed by a line passing through the arm connecting pin 14 and the bucket connecting pin 15 and a line passing through the bucket connecting pin 15 and the cutting edge 8a of the bucket 8 is defined as the bucket angle θ3. The bucket angle θ3 is calculated based on the sensor output of the stroke sensor 18C in the bucket cylinder 12.

[0030] The boom angle θ1, the arm angle θ2, and the bucket angle θ3 may be calculated using an inertial measurement unit (IMU) attached to the boom 6, the arm 7, and the bucket 8. Alternatively, the boom angle θ1, the arm angle θ2, and the bucket angle θ3 may be calculated using angle sensors attached to the boom foot pin 13, the arm connecting pin 14, and the bucket connecting pin 15. The angle sensors may be, for example, potentiometers, rotary encoders, etc.

[0031] Bucket ground angle θ B is the angle of the bucket 8 with respect to the ground G. The bucket bottom extension line Ex shown in FIG. 2 is a straight line extending the bottom surface 8b of the bucket 8 as viewed from the side. Bucket ground angle θ B is the angle between the ground surface G and the extension line Ex of the bottom surface of the bucket. When the bottom surface 8b of the bucket 8 is in a position parallel to the ground surface G, the bucket ground angle θ B When the bucket 8 is moved in the excavation direction (the direction in which the cutting edge 8a approaches the arm 7), the bucket ground angle θ B When the bucket 8 is moved in the dump direction (the direction in which the cutting edge 8a moves away from the arm 7), the bucket ground angle θ B is taken as negative.

[0032] <Outline of the construction machine control system> Next, the general configuration of the control system for the work machine will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the general configuration of the control system for the work machine shown in Fig. 1.

[0033] The control system in this embodiment includes a hydraulic excavator 100 as an example of a work machine shown in FIG. 1, and a main controller 50 and an optimization controller 60 shown in FIG.

[0034] The engine 31 is, for example, a diesel engine. An output shaft of the engine 31 is connected to a hydraulic pump 34. The engine 31 generates driving force to rotate the hydraulic pump 34. The governor 32 adjusts the amount of fuel injected by a fuel injection device in the engine 31.

[0035] The engine controller 41 outputs a command value for the fuel injection amount based on the command rotation speed of the engine 31 to the governor 32, and controls the amount of fuel injected by the fuel injection device, thereby adjusting the rotation speed of the engine 31. A rotation sensor 33 is provided on the output shaft of the engine 31. The rotation sensor 33 detects the rotation speed of the engine 31. The rotation sensor 33 outputs the detection result of the rotation speed of the engine 31 to the engine controller 41.

[0036] The hydraulic pump 34 is driven by the rotational driving force of the engine 31, and discharges pressurized oil for generating hydraulic pressure that drives the hydraulic actuators. The hydraulic actuators include hydraulic cylinders also shown in FIG. 1, namely, the boom cylinder 10, the arm cylinder 11, and the bucket cylinder 12, and hydraulic motors, namely, the travel motor 5M and the swing motor also shown in FIG. 1. The hydraulic actuators are connected to the hydraulic pump 34 via an operation valve 35.

[0037] The hydraulic pump 34 is a variable displacement hydraulic pump that has, for example, a swash plate and changes the discharge capacity by changing the tilt angle of the swash plate. A swash plate drive unit 39 is connected to the hydraulic pump 34. The swash plate drive unit 39 changes the tilt angle of the swash plate of the hydraulic pump 34. A pressure sensor 36 detects the pressure of the oil discharged from the hydraulic pump 34. The pressure sensor 36 outputs the oil pressure detection result to a pump controller 42. A portion of the oil discharged from the hydraulic pump 34 is supplied to the operation valve 35 as hydraulic oil. A portion of the oil discharged from the hydraulic pump 34 is reduced to a constant pressure by a pressure-reducing valve 37 and used as pilot oil.

[0038] The operating valve 35 is, for example, a spool-type valve that switches the direction of hydraulic oil flow by moving a rod-shaped spool. The amount of hydraulic oil supplied to the hydraulic actuator is adjusted by the axial movement of the spool. The operating valve 35 is provided with a stroke sensor 35a that detects the movement distance of the spool (spool stroke). By controlling the supply and discharge of hydraulic pressure to the hydraulic actuator, the operation of the work implement 2, the rotation of the revolving unit 3, and the traveling operation of the traveling unit 5 are controlled.

[0039] In this example, the oil supplied to the hydraulic actuator to operate the hydraulic actuator is referred to as hydraulic oil. Also, the oil supplied to the operating valve 35 to operate the spool of the operating valve 35 is referred to as pilot oil. Also, the pressure of the pilot oil is referred to as pilot oil pressure.

[0040] The hydraulic pump 34 may be one that delivers both the hydraulic oil and the pilot oil as described above. The hydraulic pump 34 may have a hydraulic pump that delivers the hydraulic oil (main hydraulic pump) and a hydraulic pump that delivers the pilot oil (pilot hydraulic pump) separately.

[0041] An EPC (electromagnetic proportional control) valve 38 is provided in the path of the pilot oil. The EPC valve 38 outputs pilot oil pressure to the operation valve 35 in accordance with a command current from the main controller 50. The operation valve 35 controls the hydraulic actuator in accordance with the pilot oil pressure.

[0042] The engine controller 41 and the pump controller 42 are mounted on the hydraulic excavator 100. The engine controller 41 and the pump controller 42 are electrically connected to each other and can transmit and receive information to and from each other. The engine controller 41 and the pump controller 42 are also electrically connected to the main controller 50.

[0043] The main controller 50 is a controller that controls the entire hydraulic excavator 100, and is configured with a CPU (Central Processing Unit), a non-volatile memory, a timer, etc. The main controller 50 controls the engine controller 41 and the pump controller 42. The main controller 50, the engine controller 41, and the pump controller 42 may be configured with one piece of hardware, or may be configured with two or more pieces of hardware.

[0044] Detection signals from the stroke sensors 18A, 18B, and 18C and the pressure sensors 10A, 10B, 11A, 11B, 12A, and 12B are input to the main controller 50. The main controller 50 may be electrically connected to each sensor by wire, or may be capable of communicating with each sensor wirelessly.

[0045] The work implement attitude acquisition unit 51 acquires the attitude of the work implement 2 based on the detection results of the stroke sensors 18A, 18B, and 18C. The work implement attitude acquisition unit 51 acquires the attitude of the work implement 2 based on the boom angle θ1, the arm angle θ2, the bucket angle θ3, and the bucket ground angle θ BWork implement attitude acquisition unit 51 acquires the position of cutting edge 8a of bucket 8 based on the calculated angles and the dimensions of boom 6, arm 7, and bucket 8. The dimensions of boom 6, arm 7, and bucket 8 are stored in memory unit 59.

[0046] The excavation volume calculation unit 52 calculates the weight Q (Figure 2) of the load loaded on the bucket 8 based on the difference between the head pressure of the boom cylinder 10 detected by the pressure sensor 10A and the bottom pressure of the boom cylinder 10 detected by the pressure sensor 10B.

[0047] An example of a method for calculating the load weight Q will be described. Fig. 4 is a schematic diagram of the work implement 2 for explaining moment balance. As shown in Fig. 4, in this embodiment, the current load weight Q in the bucket 8 is detected from the balance of each moment around the boom foot pin 13. Here, the balance of each moment around the boom foot pin 13 is expressed by the following equation (1).

[0048]

number

[0049] In equation (1), Mboomcyl is the moment around the boom foot pin 13 generated by the pressure of the hydraulic oil supplied to the boom cylinder 10. Mboom is the moment around the boom foot pin 13 due to the weight of the boom 6. Marm is the moment around the boom foot pin 13 due to the weight of the arm 7. Mbucket is the moment around the boom foot pin 13 due to the weight of the bucket 8. Q is the weight of the current load in the bucket 8. Ds is the horizontal distance from the boom foot pin 13 to the center of gravity C4 of the load in the bucket 8. Here, the center of gravity C4 of the load when a load of the rated load is loaded in the bucket 8 is stored in the memory unit 59 of the main controller 50. Q×Ds is the moment around the boom foot pin 13 due to the load in the bucket 8.

[0050] Mboomcyl is calculated from the load on the boom cylinder 10 (head pressure and bottom pressure, i.e., the pressure of the hydraulic oil supplied to the boom cylinder 10). The head pressure of the boom cylinder 10 is detected by pressure sensor 10A. The bottom pressure of the boom cylinder 10 is detected by pressure sensor 10B. The excavated soil volume calculation unit 52 calculates the moment Mboomcyl around the boom foot pin 13 generated by the load on the boom cylinder 10 based on the head pressure and bottom pressure of the boom cylinder 10.

[0051] Mboom is calculated by multiplying the distance r1 between the center of gravity C1 of the boom 6 and the boom foot pin 13 by the weight M1 of the boom 6 (r1 × M1). The position of the center of gravity C1 of the boom 6 is calculated from the boom angle θ1 and other factors. The weight M1 of the boom 6 and other factors are stored in the memory unit 59.

[0052] Marm is calculated as the product (r2 × M2) of the distance r2 between the center of gravity C2 of the arm 7 and the boom foot pin 13 and the weight M2 of the arm 7. The position of the center of gravity C2 of the arm 7 is calculated from the arm angle θ2 and other factors. The weight M2 of the arm 7 and other factors are stored in the memory unit 59.

[0053] Mbucket is calculated as the product (r3 × M3) of the distance r3 between the center of gravity C3 of the bucket 8 and the boom foot pin 13 and the weight M3 of the bucket 8. The position of the center of gravity C3 of the bucket 8 is calculated from the bucket angle θ3 and the like. The weight M3 of the bucket 8 and the like are stored in the memory unit 59.

[0054] In calculating the weight Q of the load in the bucket 8, the excavated soil volume calculation unit 52 calculates the positions of the centers of gravity C1, C2, C3, and C4 based on the boom angle θ1, arm angle θ2, and bucket angle θ3 calculated by the work machine attitude acquisition unit 51. The excavated soil volume calculation unit 52 calculates the distances r1, r2, and r3 between the centers of gravity C1, C2, and C3 and the boom foot pin 13.

[0055] Excavation volume calculation unit 52 reads out the weight M1 of boom 6 from memory unit 59, and calculates the product of distance r1 and weight M1 as moment Mboom of boom 6 about boom foot pin 13. Excavation volume calculation unit 52 reads out the weight M2 of arm 7 from memory unit 59, and calculates the product of distance r2 and weight M2 as moment Marm of arm 7 about boom foot pin 13. Excavation volume calculation unit 52 reads out the weight M3 of bucket 8 from memory unit 59, and calculates the product of distance r3 and weight M3 as moment Mbucket of bucket 8 about boom foot pin 13.

[0056] The excavated soil volume calculation unit 52 reads the dimensions of the boom 6, the dimensions of the arm 7, and the center of gravity C4 of the load in the bucket 8 at the rated load from the memory unit 59. The excavated soil volume calculation unit 52 calculates the horizontal distance Ds from the boom foot pin 13 to the center of gravity C4 of the load based on the boom angle θ1, the arm angle θ2, the bucket angle θ3, the dimensions of the boom 6 and the arm 7, and the center of gravity C4 of the load.

[0057] The excavation volume calculation unit 52 substitutes the moments Mboomcyl, Mboom, Marm, Mbucket and distance Ds calculated as above into the above equation (1). In this way, the excavation volume calculation unit 52 calculates the weight Q of the load loaded in the bucket 8. The weight Q of the load in the bucket 8 is calculated based on the load on the boom cylinder 10 and the attitude of the work implement 2.

[0058] Returning to FIG. 3 , the workload calculation unit 53 calculates the workload required for excavation work to excavate an excavation target with the bucket 8. Specifically, the workload calculation unit 53 acquires the head pressure of the boom cylinder 10 detected by pressure sensor 10A and the bottom pressure of the boom cylinder 10 detected by pressure sensor 10B. The workload calculation unit 53 reads out the dimensions or specifications of the boom cylinder 10 from the memory unit 59. The workload calculation unit 53 acquires the inner diameter of the cylinder and the diameter of the cylinder rod of the boom cylinder 10. The workload calculation unit 53 acquires the bore area of ​​the cylinder and the cross-sectional area of ​​the cylinder rod.

[0059] The workload calculation unit 53 calculates the thrust of the boom cylinder 10 using the following equation (2): In equation (2), Fboomcyl is the thrust of the boom cylinder 10, Pb is the bottom pressure of the boom cylinder 10, Ph is the head pressure of the boom cylinder 10, S is the cylinder bore area of ​​the boom cylinder 10, and D is the cross-sectional area of ​​the cylinder rod of the boom cylinder 10.

[0060]

number

[0061] Similarly, the work amount calculation unit 53 calculates the thrust of the arm cylinder 11 and the thrust of the bucket cylinder 12. As shown in Fig. 1, the work implement 2 has two boom cylinders 10. The work amount calculation unit 53 calculates the amount of work during excavation work by integrating the integrated values ​​of the thrust of the two boom cylinders 10, the thrust of the arm cylinder 11, and the thrust of the bucket cylinder 12 from the start to the end of the excavation work.

[0062] The work implement control unit 58 controls the work implement 2, which performs excavation work with the bucket 8. A command current is output from the work implement control unit 58 to the EPC valve 38. In accordance with this command current, the EPC valve 38 outputs pilot oil pressure to the operation valve 35. In accordance with this pilot oil pressure, the operation valve 35 supplies a predetermined amount of hydraulic oil to each of the boom cylinder 10, the arm cylinder 11, and the bucket cylinder 12. The boom cylinder 10 extends and retracts, causing the boom 6 to move up and down. The arm cylinder 11 extends and retracts, causing the arm 7 to move in the excavation direction (towards the revolving unit 3) and the dumping direction (away from the revolving unit 3). The bucket cylinder 12 extends and retracts, causing the bucket 8 to move in the excavation direction and the dumping direction.

[0063] The spool of the operating valve 35 is not limited to a type that is operated by supplying pilot hydraulic pressure, but may be a solenoid-driven type. The work implement control unit 58 may control the work implement 2 by outputting a control signal to the solenoid-driven spool to control the position of the spool.

[0064] The optimization controller 60 is electrically connected to the main controller 50. The optimization controller 60 adjusts control parameters for automatically controlling the excavation work performed by the hydraulic excavator 100.

[0065] The optimization controller 60 may be mounted on the hydraulic excavator 100. The optimization controller 60 may be connected to the main controller 50 by wire.

[0066] The optimization controller 60 may be installed outside the hydraulic excavator 100. The optimization controller 60 may be wirelessly connected to the main controller 50. The optimization controller 60 may be located at the work site of the hydraulic excavator 100, or may be located in a remote location away from the work site of the hydraulic excavator 100. The optimization controller 60 may be a portable device. The optimization controller 60 may be a portable device that can be carried and used by an operator, such as a laptop computer, a tablet computer, or a smartphone.

[0067] <Automatic control of excavation work, control parameters> 5 is a diagram showing the operation of the work machine 2 during automatic excavation. Before automatic excavation begins, the work machine 2 is manually set to an initial position. The initial position is a position in which the length of the arm cylinder 11 is minimized, the arm 7 is farthest from the rotating body 3, and the cutting edge 8a of the bucket 8 contacts the ground G. In the initial position, the cutting edge 8a of the bucket 8 may be located on a straight line passing through the arm connecting pin 14 and the bucket connecting pin 15 in a side view.

[0068] As the arm 7 moves in the excavation direction from the initial position and the bucket 8 moves in the excavation direction, a penetration sweeping operation is performed in which the cutting edge 8a of the bucket 8 penetrates the ground and scoops up earth and sand into the bucket 8, as shown in (a) of Figure 5. At this time, the arm 7 moves in the same way as when the amount of operation of the arm operating lever is constant. The arm 7 may also move in the same way as when the amount of operation of the arm operating lever for moving the arm 7 in the excavation direction is maximum.

[0069] Bucket ground angle θ B The penetration sweeping operation continues until the bucket ground angle θ becomes greater than the threshold value TH_BK1 or the arm angle θ2 becomes smaller than the threshold value TH_A. B The threshold value TH_BK1 for the arm angle θ2 may be 0°. The threshold value TH_A for the arm angle θ2 may be 80°.

[0070] Bucket ground angle θ B becomes larger than the threshold value TH_BK1, or the arm angle θ2 becomes smaller than the threshold value TH_A, a scooping operation is performed in which the bucket 8 is moved in the excavation direction while the boom 6 is raised, as shown in (b) of FIG. 5.

[0071] The cutting edge height refers to the vertical position of the cutting edge 8a of the bucket 8 relative to the boom foot pin 13. When the cutting edge 8a and the boom foot pin 13 are in the same vertical position, the cutting edge height is defined as 0. The cutting edge height is considered positive when the cutting edge 8a is located higher in the vertical direction than the boom foot pin 13 and is farther from the ground G than the boom foot pin 13. The cutting edge height is considered negative when the cutting edge 8a is located lower in the vertical direction than the boom foot pin 13 and is closer to the ground G than the boom foot pin 13.

[0072] Bucket ground angle θ B The scooping operation continues until the bucket angle θ becomes greater than the threshold value TH_BK2 or the blade height becomes greater than the threshold value TH_H. BThe threshold value TH_BK2 for the cutting edge height may be 40°. The threshold value TH_H for the cutting edge height may be −1.0 m.

[0073] Bucket ground angle θ B becomes larger than the threshold value TH_BK2 or the cutting edge height becomes larger than the threshold value TH_H, the excavation work under automatic control is terminated.

[0074] The weight of the load to be loaded into the bucket 8 during this automatic excavation is calculated by the excavation volume calculation unit 52. The workload during automatic excavation is calculated by the workload calculation unit 53. The calculated weight of the load and workload are output from the main controller 50 to the optimization controller 60.

[0075] The optimization controller 60 adjusts the control parameters x for automatically controlling the excavation operation. Five control parameters x shown in Fig. 5 are set.

[0076] The first parameter is the "arm bottom pressure threshold for load release." The second parameter is the "boom lever operation amount for load release." If the arm 7 becomes stuck during excavation, for example because the bucket 8 gets caught on hard ground, the bottom pressure of the arm cylinder 11 increases. When the bottom pressure of the arm cylinder 11 increases to a predetermined threshold, the boom 6 is raised to release the arm 7 from the stuck state. As the arm 7 returns to a movable state, the bottom pressure of the arm cylinder 11 decreases.

[0077] At this time, the boom 6 operates in the same manner as when the amount of operation of the boom operation lever is constant. The control parameter x includes the condition for starting the raising operation of the boom 6 during the penetration sweeping operation and the amount of operation of the boom operation lever when the boom 6 is raised.

[0078] The third parameter is the "bucket lever operation amount" during the penetration sweeping operation. During the penetration sweeping operation, the bucket 8 operates in the excavation direction. At this time, the bucket 8 operates in the same manner as when the operation amount of the bucket operation lever is constant. The control parameter x includes the operation amount of the bucket operation lever when the bucket 8 performs an excavation operation during the penetration sweeping operation.

[0079] The fourth parameter is the "boom lever operation amount" during the scooping operation. The fifth parameter is the "bucket lever operation amount" during the scooping operation. During the scooping operation, the boom 6 performs a raising operation, and the bucket 8 performs a digging operation. At this time, the boom 6 performs the same operation as when the operation amount of the boom operation lever is constant, and the bucket 8 performs the same operation as when the operation amount of the bucket operation lever is constant. The control parameter x includes the operation amount of the boom operation lever when the boom 6 performs a raising operation during the scooping operation, and includes the operation amount of the bucket operation lever when the bucket 8 performs a digging operation.

[0080] The control parameter x includes a command value for operating the work machine actuator, which is output to the work machine actuator that drives the work machine 2. The control parameter x includes a command value for operation of the work machine actuator during excavation work. The control parameter x includes an operation amount of an operating lever for operating the work machine 2 during excavation work. The control parameter x includes a set value for the flow rate of hydraulic oil supplied to the work machine actuator during excavation work. The control parameter x includes a set value for the speed at which the work machine actuator operates during excavation work. If the work machine actuator is a hydraulic cylinder, the control parameter x includes a set value for the extension / retraction speed of the hydraulic cylinder during excavation work.

[0081] The control parameter x includes a set value for the amount of change in the angle of the boom 6 relative to the rotating structure 3 during excavation work. The control parameter x includes a set value for the amount of change in the angle of the bucket 8 relative to the arm 7 during excavation work.

[0082] The optimization controller 60 adjusts the control parameter x using an evaluation function expressed as a linear combination of multiple indexes including the weight of the load loaded into the bucket 8 during excavation work and the cost of the excavation work. The optimization controller 60 may adjust the control parameter x so as to maximize or minimize the evaluation function. The evaluation function is sometimes referred to as an objective function. The cost includes the amount of work required for the excavation work.

[0083] The cost includes the workload of the work equipment actuator. The cost may include the workload (discharge pressure and discharge flow rate) of the hydraulic pump 34 (FIG. 2). The cost may include fuel consumption (work workload of the engine 31). The workload required for excavation work includes at least one of the workload of the work equipment actuator, the workload of the hydraulic pump 34, and the workload of the engine 31. The cost may include the magnitude of sway of the vehicle body (swinging unit 3). The cost may include the number of times hydraulic pressure is relieved. If the traveling body 5 has tires instead of tracks 5Cr, the cost may include the number of times the tires slip.

[0084] The cost is an index that is desired to be reduced in the excavation work. The evaluation function may include multiple costs. A different weighting coefficient may be set for each of the multiple costs.

[0085] In this embodiment, the evaluation function is expressed by the following equation (3). In equation (3), E(x) is the evaluation function, Q(x) is the weight of the load loaded into the bucket 8 during excavation work, w is a weighting coefficient, and P(x) is the amount of work from the start to the end of excavation. The evaluation function E(x) is a function of the control parameter x. The load weight Q(x) is a function of the control parameter x. The amount of work P(x) is a function of the control parameter x.

[0086]

number

[0087] The evaluation function E(x) is expressed as the linear sum of the load weight Q(x) as the reward for excavation work and the workload P(x) as the cost of excavation work. The weighting coefficient w is an arbitrarily set value. When the weighting coefficient w is small, the load weight Q(x) is emphasized, and the control parameter x is adjusted to increase the load weight Q(x). When the weighting coefficient w is large, the workload P(x) is emphasized, and the control parameter x is adjusted to decrease the workload P(x). By changing the weighting coefficient w, it is possible to change the balance between the load weight Q(x) and workload P(x), which are in a trade-off relationship.

[0088] The optimization controller 60 outputs the adjusted control parameter x to the main controller 50. The work machine control unit 58 of the main controller 50 uses the input control parameter x to perform excavation work under automatic control. The optimization controller 60 adjusts the control parameter x for automatically controlling the excavation work, the main controller 50 (work machine control unit 58) performs automatic excavation, and the main controller 50 (work machine control unit 58) calculates the load weight Q(x) and the workload P(x) during automatic excavation, and these steps are repeated to determine the appropriate control parameter x.

[0089] <Bayesian optimization> Because the shape and properties of the objects excavated by work machines change continuously, control parameters must also be continuously adapted when performing automatic excavation. It is necessary to automatically search for the most productive excavation method for the site conditions with minimal test excavation. To obtain the optimal solution without exhaustive test excavation, it is necessary to strike a balance between improving the provisional solution obtained at a given point in time and exploring unknown areas.

[0090] In this embodiment, the optimization controller 60 uses Bayesian optimization to obtain control parameters x that maximize an evaluation function E(x), which is composed of multiple indexes including the load weight Q(x) and the workload P(x). Bayesian optimization is a type of "experimental design" that optimizes experimental conditions from a relatively small number of experimental data sets by alternately performing experiments, model estimation from experimental data, searching for optimal solutions, and determining the next experimental conditions.

[0091] Bayesian optimization is a method for estimating the expected value and uncertainty of an unknown function through repeated trials. Bayesian optimization is composed of Gaussian process regression, which estimates the expected value and variance of a function from the trial data set obtained up to that point, and an algorithm that determines the next trial conditions based on this. It can be used as a type of dynamic experimental design. In the process of adjusting the control parameters, the optimization controller 60 determines the next control parameter x to be evaluated, taking into account both the predicted mean value of the evaluation function E(x) and the uncertainty in the prediction of the evaluation function E(x).

[0092] A confidence upper bound function (UCB) is used to search for the optimal control parameter x while taking into account the uncertainty of the evaluation function E(x). The confidence upper bound function in Bayesian optimization is an example of an acquisition function for determining the next point to be evaluated in the optimization process. The confidence upper bound function determines the next point to be evaluated by taking into account both the predicted value of the evaluation function and its uncertainty (variance). Specifically, the confidence upper bound function is expressed by the following equation (4). In equation (4), μ(ξ) is the predicted mean value of the evaluation function at point ξ, σ(ξ) is the standard deviation (uncertainty) of the prediction at point ξ, and β is an adjustment parameter for adjusting the balance between exploration and exploitation.

[0093]

number

[0094] The reliability upper bound function strikes a balance between utilizing known areas with high predicted mean values ​​and exploring unknown areas with high uncertainty. This balance is adjusted by the value of the adjustment parameter β. Increasing the adjustment parameter β places emphasis on reducing uncertainty (variance), while decreasing the adjustment parameter β places emphasis on more detailed exploration of known areas. The adjustment parameter β allows for flexible adjustment of the tendency between exploration and exploitation. By actively exploring areas with high uncertainty, the risk of falling into a local optimum is reduced.

[0095] <Optimization flow of control parameter x for automatic excavation> 6 is a flowchart showing the flow of processing for optimizing the control parameter x for automatic excavation. The optimization controller 60 optimizes the control parameter x for automatically controlling the excavation work in accordance with the flow of processing shown in FIG.

[0096] As shown in FIG. 6, in step S1, an initial data set is prepared. The initial data set includes at least one data item indicating the results of excavation attempts that have already been made. Each data item includes a control parameter x when excavation was attempted and a value of the evaluation function E(x) corresponding to that control parameter x. Each data item indicates the relationship between input and output, i.e., the result of the evaluation function E(x) when the control parameter x during automatic excavation was set to a certain value. The initial data set may include any number of data items, but preferably includes multiple data items. The optimization controller 60 acquires the initial data set.

[0097] In step S2, the optimization controller 60 estimates a Gaussian process model of the evaluation function E(x) using the initial data set.

[0098] 7A and 7B are diagrams showing a first example of a Gaussian process model. The horizontal axis of the graph shown in Fig. 7A is a set of control parameters x, and the vertical axis is an evaluation function E(x). The horizontal axis of the graph shown in Fig. 7B is a set of control parameters x, and the vertical axis is an acquisition function, specifically, a reliability upper limit function. The set of control parameters x includes the first to fifth parameters shown in Fig. 5.

[0099] The two crosses shown in the graph of FIG. 7(A) represent plots of the sets of control parameters x and the evaluation function E(x) corresponding to the two pieces of data included in the initial data set. The optimization controller 60 uses the two pieces of data previously acquired to estimate the predicted average value of the evaluation function E(x) and the prediction uncertainty of the evaluation function E(x). The curve shown in FIG. 7(A) represents the predicted average value of the evaluation function E(x). The hatched area shown in FIG. 7(A) represents the prediction uncertainty (variance) of the evaluation function E(x).

[0100] In step S3, the optimization controller 60 calculates the control parameters x for the next trial that maximize the evaluation value according to the reliability upper limit function (UCB). The optimization controller 60 creates the graph shown in FIG. 7(B) using the predicted average value of the evaluation function E(x) estimated as shown in FIG. 7(A) and the prediction uncertainty of the evaluation function E(x), as well as the above-mentioned equation (4). The optimization controller 60 calculates the set of control parameters x that maximizes the acquisition function (reliability upper limit function) shown in FIG. 7(B) as the set of control parameters x to be used in the next trial of automatic excavation. The white circles shown in FIG. 7(A) indicate the control parameters x for the next trial. The optimization controller 60 outputs the control parameters x for the next trial to the main controller 50.

[0101] In step S4, the main controller 50 executes a trial of automatic excavation using the set of control parameters x calculated by the optimization controller 60. The work implement control unit 58 outputs a command current based on the set of control parameters x input from the optimization controller 60 to the EPC valve 38. In accordance with the command current, the EPC valve 38 outputs pilot oil pressure to the operation valve 35. In accordance with this pilot oil pressure, the operation valve 35 supplies a predetermined amount of hydraulic oil to each of the boom cylinder 10, the arm cylinder 11, and the bucket cylinder 12. The work implement 2 operates, and a trial of excavation work is executed using automatic control.

[0102] The excavation volume calculation unit 52 calculates the weight Q(x) of the load to be loaded on the bucket 8 during the trial excavation work. The work amount calculation unit 53 calculates the work amount P(x) during the excavation work. The main controller 50 outputs the calculated load weight Q(x) and work amount P(x) to the optimization controller 60.

[0103] In step S5, the optimization controller 60 calculates the value of the evaluation function E(x) by substituting the input load weight Q(x) and work amount P(x) into equation (3).

[0104] In step S6, the optimization controller 60 adds the set of control parameters x and the calculation result of the evaluation function E(x) corresponding to the set of control parameters x to the data set.

[0105] In step S7, the optimization controller 60 determines whether the number of attempts of automatic excavation performed so far has reached a predetermined maximum number of attempts. The optimization controller 60 also determines whether the value of the evaluation function E(x) calculated in step S5 has reached a predetermined target performance.

[0106] If the number of trials has not reached the maximum number of trials and the evaluation function E(x) has not reached the target performance (NO in step S7), the process returns to step S2. The optimization controller 60 estimates a Gaussian process model of the evaluation function E(x) using the data set to which the new evaluation results have been added.

[0107] Figure 8 shows a second example of a Gaussian process model. As in Figure 7, the horizontal axis of the graph shown in Figure 8(A) represents the set of control parameters x, and the vertical axis represents the evaluation function E(x). The horizontal axis of the graph shown in Figure 8(B) represents the set of control parameters x, and the vertical axis represents the acquisition function (reliability upper limit function).

[0108] The multiple x's shown in the graph of FIG. 8(A) represent plots of the sets of control parameters x and the evaluation function E(x) included in the data set to which evaluation results from previous trials have been added. The optimization controller 60 estimates the predicted average value of the evaluation function E(x) and the prediction uncertainty of the evaluation function E(x) using the data acquired so far. The curve shown in FIG. 8(A) indicates the predicted average value of the evaluation function E(x). The hatched area shown in FIG. 8(A) indicates the prediction uncertainty (variance) of the evaluation function E(x).

[0109] The optimization controller 60 creates the graph shown in Figure 8(B) using the predicted average value of the evaluation function E(x) estimated as shown in Figure 8(A) and the prediction uncertainty of the evaluation function E(x), as well as the above-mentioned equation (4). The optimization controller 60 calculates the set of control parameters x that maximizes the acquisition function (reliability upper limit function) shown in Figure 8(B) as the set of control parameters x to be used in the next trial of automatic excavation. The white circles shown in Figure 8(A) indicate the control parameters x for the next trial.

[0110] If the number of trials reaches the maximum number of trials or the evaluation function E(x) reaches the target performance (YES in step S7), the process proceeds to step S8, where the optimization controller 60 ends the optimization of the control parameter x.

[0111] <Bayesian optimization results> Fig. 9 is a diagram showing the convergence curve of the evaluation function E(x). The horizontal axis of Fig. 9 represents the number of trials of automatic excavation, and the vertical axis represents the evaluation function E(x). The square marks in Fig. 9 represent plots of the values ​​of the evaluation function E(x) for each trial. The broken line in Fig. 9 represents the maximum point set of the evaluation function E(x).

[0112] Figure 9 shows that the evaluation function E(x) converges sufficiently after about 20 trials. By using Bayesian optimization, the control parameter x for automatically controlling excavation work can be adjusted with a small number of trials, which is useful for saving money and time.

[0113] FIG. 10 is a diagram showing the relationship between the amount of soil and the amount of work for each weighting coefficient w. The horizontal axis of the graph shown in FIG. 10 is the weight Q(x) (amount of soil) of the load loaded into the bucket 8 during excavation work, and the vertical axis is the amount of work P(x) from the start to the end of excavation. The circles in FIG. 10 represent plots of the values ​​of the weight Q(x) of the load and the amount of work P(x) when automatic excavation is performed using the calculated control parameter x, calculated by changing the weighting coefficient w shown in equation (3) above. FIG. 11 is a diagram showing the optimal solution for the control parameter x for each weighting coefficient w.

[0114] As shown in Figures 10 and 11, when the weighting coefficient w is reduced, excavation is performed with emphasis on the amount of soil. When the weighting coefficient w is reduced, during the penetration sweeping operation, the boom 6 is not raised very much and excavation is performed by the movement of the arm 7 and bucket 8, so that a large amount of the excavation target is scooped up into the bucket 8. In order not to spill the scooped excavation target, the speed of the boom 6 during the scooping operation is reduced and the bucket 8 is raised slowly. This increases the amount of soil. In this case, the excavation time becomes longer, and the workload increases.

[0115] When the weight coefficient w is increased, excavation that emphasizes the amount of work is performed. When the weight coefficient w is increased, the excavation target is dug shallowly so that the bucket 8 is not subjected to a large load, resulting in a shorter excavation time, a smaller amount of work, and a smaller amount of soil.

[0116] By changing the weight coefficient w, it is possible to converge to excavation control with a good balance between the load weight Q(x) and the work amount P(x). Among the multiple indicators included in the evaluation function E(x), which indicator to emphasize can be adjusted by weighting. The weight coefficient w can be changed so that the weighting of the indicators to be emphasized is increased and the weighting of the indicators not to be emphasized is decreased among the multiple indicators included in the evaluation function E(x). A user (for example, an in-house tester, a field supervisor, or an operator) who optimizes the control parameter x can determine the target value of the weight coefficient w based on the relationship between the soil amount and the work amount shown in FIG. 10.

[0117] Depending on the site conditions, the optimal relationship between the soil amount and the work amount is different. When the type of working machine for excavating the excavation target changes or the soil changes, the optimal excavation method will change. It is required to optimally adjust the control parameter x according to the site conditions. By applying Bayesian optimization to the adjustment of the control parameter x, the control parameter x for automatically controlling the excavation work can be appropriately adjusted with a small number of trial times.

[0118] <Introduction of Safe Opt> Safe Exploration for Optimization (Safe Opt) has been studied as a method that can efficiently estimate a safe control parameter x while suppressing the deterioration of the evaluation function E(x). Safety here means that the work amount P(x) does not increase rapidly. By applying this method, it is expected that the optimal solution of the control parameter x for highly productive automatic excavation can be explored while preventing the increase of the work amount P(x). Hereinafter, the optimization of the control parameter x for automatic excavation when Safe Opt is introduced will be described.

[0119] FIG. 12 is a flowchart showing the flow of processing for optimizing the control parameter x when Safe Opt is introduced.

[0120] As shown in Fig. 12, in step S1, an initial data set is prepared. The initial data set includes a combination of initial trial points and their corresponding evaluation values, and an initial safety index. The optimization controller 60 acquires the initial data set. In the example described here, the initial data set does not include results of excavations that have already been attempted.

[0121] Next, in step S11, the optimization controller 60 estimates a Gaussian process model of the safety index using the initial data set.

[0122] Figure 13 shows a third example of a Gaussian process model. The horizontal axis of the graph shown in Figure 13(A) is the set of control parameters x, and the vertical axis is the evaluation function E(x). The horizontal axis of the graph shown in Figure 13(B) is the set of control parameters x, and the vertical axis is the safety index for excavation work. The horizontal axis of the graph shown in Figure 13(C) is the set of control parameters x, and the vertical axis is the acquisition function (reliability upper limit function).

[0123] The graph shown in Figure 13(B) is a Gaussian process model of the safety index. The safety index may be the amount of work required for the excavation work. The safety index may be an index included in the cost required for the excavation work. The safety index may be one of multiple indexes included in the evaluation function E(x).

[0124] The safety index may be an index not included in the evaluation function. For example, the possibility of the work machine tipping over, mechanical damage such as the load on the engine 31 or the load on the hydraulic pump 34, etc. may be used as the safety index.

[0125] In the graph of Figure 13(A), crosses are plotted to indicate a set of initial control parameters x and the corresponding initial values ​​of the evaluation function E(x). In the graph of Figure 13(B), crosses are plotted to indicate a set of initial control parameters x and the corresponding initial values ​​of the safety index. The value of the initial set of control parameters x is zero, and the corresponding initial values ​​of the evaluation function E(x) and the safety index are also zero. The initial values ​​of the predicted average values ​​of the evaluation function E(x) and the safety index are zero.

[0126] The optimization controller 60 estimates the predicted mean value of the safety index and the uncertainty of the prediction of the safety index using the initial set of control parameters x and the initial value of the safety index. The solid line shown in Figure 13(B) indicates the predicted mean value of the safety index. The hatched area shown in Figure 13(B) indicates the uncertainty (variance) of the prediction of the safety index.

[0127] The graph in Figure 13(B) also shows a threshold value TH_SAFE of the safety index as a dashed line. The threshold value TH_SAFE is stored in an arbitrary storage device such as the storage unit 59 or the memory of the optimization controller 60. The optimization controller 60 reads out the threshold value TH_SAFE and draws a dashed line indicating the threshold value TH_SAFE in the graph in Figure 13(B).

[0128] 12, in step S12, the optimization controller 60 calculates an area equal to or greater than the safety index threshold TH_SAFE, and sets this as the safety area Sn. The optimization controller 60 estimates the safety area Sn based on the uncertainty of the safety index prediction and the safety index threshold TH_SAFE. The optimization controller 60 sets the area in which the lower limit of the uncertainty of the safety index prediction is equal to or greater than the safety index threshold TH_SAFE as the safety area Sn.

[0129] In step S13, the optimization controller 60 identifies a set Mn of control parameters x that are within the safety region Sn and have the potential to maximize the performance function E(x).

[0130] In step S14, the optimization controller 60 identifies a set Gn of control parameters x that are outside the safety region Sn and have the potential to expand the safety region Sn.

[0131] In step S15, the optimization controller 60 sets the region included in the set Mn and the set Gn as the region for searching for the next control parameter to be evaluated.

[0132] In step S2, the optimization controller 60 estimates a Gaussian process model of the evaluation function E(x) using the initial data set. The optimization controller 60 estimates the predicted mean value of the evaluation function E(x) and the prediction uncertainty of the evaluation function E(x) using the initial set of control parameters x and the initial value of the evaluation function E(x). The solid line shown in FIG. 13(A) indicates the predicted mean value of the evaluation function E(x). The hatched area shown in FIG. 13(A) indicates the prediction uncertainty (variance) of the evaluation function E(x).

[0133] In step S3, the optimization controller 60 calculates the control parameter x for the next trial that maximizes the evaluation value according to the reliability upper limit function (UCB). In this example, the optimization controller 60 creates a graph of the acquisition function shown in FIG. 13(C) with the goal of expanding the safety region Sn. The optimization controller 60 calculates the set of control parameters x that maximizes the acquisition function (reliability upper limit function) as the set of control parameters x to be used in the next trial of automatic excavation. The open circle shown in FIG. 13(B) indicates the control parameter x for the next trial. The control parameter x for the next trial is outside the safety region Sn. The optimization controller 60 outputs the control parameter x for the next trial to the main controller 50.

[0134] The subsequent processing of steps S4 to S6 is performed in the same manner as in Fig. 6. If it is determined in step S7 that the number of trials has not reached the maximum number of trials and that the evaluation function E(x) has not reached the target performance (NO in step S7), the processing returns to step S11. The optimization controller 60 estimates a Gaussian process model of the evaluation function E(x) and a Gaussian process model of the safety index using the data set to which the new evaluation results have been added.

[0135] Figure 14 is a diagram showing a fourth example of a Gaussian process model. As in Figure 13, the horizontal axis of the graph shown in Figure 14(A) represents the set of control parameters x, and the vertical axis represents the evaluation function E(x). The horizontal axis of the graph shown in Figure 14(B) represents the set of control parameters x, and the vertical axis represents the safety index for excavation work. The horizontal axis of the graph shown in Figure 14(C) represents the set of control parameters x, and the vertical axis represents the acquisition function (reliability upper limit function).

[0136] The multiple crosses shown in the graphs of Figures 14(A) and 14(B) represent plots of the evaluation function E(x) and safety index corresponding to the set of control parameters x included in the data set to which evaluation results from previous trials have been added.

[0137] The optimization controller 60 estimates the predicted average value of the evaluation function E(x) and the prediction uncertainty of the evaluation function E(x) using data that has been acquired so far. The curve shown in FIG. 14(A) indicates the predicted average value of the evaluation function E(x). The hatched area shown in FIG. 14(A) indicates the prediction uncertainty (variance) of the evaluation function E(x). The optimization controller 60 estimates the predicted average value of the safety index and the prediction uncertainty of the safety index using data that has been acquired so far. The curve shown in FIG. 14(B) indicates the predicted average value of the safety index. The hatched area shown in FIG. 14(B) indicates the prediction uncertainty (variance) of the safety index.

[0138] The optimization controller 60 creates a graph of the acquisition function shown in FIG. 14(C) with the goal of further expanding the safety region Sn. The optimization controller 60 calculates the set of control parameters x that maximizes the acquisition function (reliability upper limit function) as the set of control parameters x to be used in the next trial of automatic excavation. The white circles shown in FIG. 14(B) indicate the control parameters x for the next trial. The control parameters x for the next trial are outside the safety region Sn. The optimization controller 60 outputs the control parameters x for the next trial to the main controller 50.

[0139] Figure 15 is a diagram showing a fifth example of a Gaussian process model. As in Figure 13, the horizontal axis of the graph shown in Figure 15(A) represents the set of control parameters x, and the vertical axis represents the evaluation function E(x). The horizontal axis of the graph shown in Figure 15(B) represents the set of control parameters x, and the vertical axis represents the safety index for excavation work. The horizontal axis of the graph shown in Figure 15(C) represents the set of control parameters x, and the vertical axis represents the acquisition function (reliability upper limit function).

[0140] The multiple crosses shown in the graphs of Figures 15(A) and 15(B) represent plots of the evaluation function E(x) and safety index corresponding to the set of control parameters x included in the data set to which evaluation results from previous trials have been added.

[0141] The optimization controller 60 estimates the predicted average value of the evaluation function E(x) and the prediction uncertainty of the evaluation function E(x) using data that has been acquired so far. The curve shown in FIG. 15(A) indicates the predicted average value of the evaluation function E(x). The hatched area shown in FIG. 15(A) indicates the prediction uncertainty (variance) of the evaluation function E(x). The optimization controller 60 estimates the predicted average value of the safety index and the prediction uncertainty of the safety index using data that has been acquired so far. The curve shown in FIG. 15(B) indicates the predicted average value of the safety index. The hatched area shown in FIG. 15(B) indicates the prediction uncertainty (variance) of the safety index.

[0142] The optimization controller 60 creates a graph of the acquisition function shown in FIG. 15(C) with the goal of maximizing the evaluation function E(x). The optimization controller 60 calculates the set of control parameters x that maximizes the acquisition function (reliability upper limit function) as the set of control parameters x to be used in the next trial of automatic excavation. The white circles shown in FIG. 15(A) indicate the control parameters x for the next trial. The control parameters x for the next trial are within the safety region Sn. The optimization controller 60 outputs the control parameters x for the next trial to the main controller 50.

[0143] If the number of trials reaches the maximum number of trials or the evaluation function E(x) reaches the target performance (YES in step S7), the process proceeds to step S8, where the optimization controller 60 ends the optimization of the control parameter x.

[0144] In the explanation so far, a hydraulic excavator 100 has been given as an example of a work machine. The work machine is not limited to the hydraulic excavator 100 having a backhoe-type work implement 2 in which the opening of the bucket 8 faces rearward. The work machine may also be a loading shovel. The work machine may also be a wheel loader. The work implement actuator that drives the work implement is not limited to a hydraulic type, but may also be an electric type, a pneumatic type, or the like.

[0145] <Additional Notes> The above description includes the following additional features.

[0146] (Appendix 1) a work machine having a bucket at a tip thereof and performing excavation work with the bucket; a controller for adjusting control parameters for controlling the excavation operation; The controller adjusting the control parameters using an evaluation function that is a function of the control parameters and includes a weight of a load to be loaded on the bucket during the excavation operation and a cost required for the excavation operation; A work machine control system that, in the process of adjusting the control parameters, determines the control parameters to be evaluated next by taking into consideration both the predicted average value of the evaluation function and the prediction uncertainty of the evaluation function.

[0147] (Appendix 2) 2. A work machine control system as described in Appendix 1, wherein the cost includes the amount of work required for the excavation operation.

[0148] (Appendix 3) 3. The control system for a work machine according to claim 1, wherein the controller adjusts which of the multiple indexes included in the evaluation function is to be emphasized by weighting.

[0149] (Appendix 4) Further provided is a work machine actuator that drives the work machine, 4. A work machine control system according to any one of Supplementary Note 1 to Supplementary Note 3, wherein the control parameters include a command value for operation of the work machine actuator during the excavation operation.

[0150] (Appendix 5) 5. The work machine control system according to claim 1, wherein the control parameter includes an amount of change in angle of the bucket during the excavation operation.

[0151] (Appendix 6) The control system for a work machine according to any one of Supplementary Note 1 to Supplementary Note 5, wherein the controller estimates a safety area based on a safety index for the excavation work and a threshold value for the safety index, and determines the control parameter outside the safety area as the control parameter to be evaluated next, with the aim of expanding the safety area.

[0152] (Appendix 7) 7. A work machine control system as described in Appendix 6, wherein the controller determines the control parameter that is within the safety region as the control parameter to be evaluated next, with the goal of maximizing or minimizing the evaluation function.

[0153] (Appendix 8) 8. The work machine control system of claim 6 or 7, wherein the controller estimates the safety region based on the uncertainty of a prediction of the safety index and the threshold value of the safety index.

[0154] (Appendix 9) 9. The control system for a work machine according to any one of Supplementary Note 6 to Supplementary Note 8, wherein the controller sets one index out of a plurality of indexes included in the evaluation function as the safety index.

[0155] The embodiments disclosed herein are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0156] 1 main body, 2 work equipment, 3 rotating body, 5 traveling body, 6 boom, 7 arm, 8 bucket, 8a cutting edge, 8b bottom, 10 boom cylinder, 10A, 10B, 11A, 11B, 12A, 12B pressure sensor, 11 arm cylinder, 12 bucket cylinder, 13 boom foot pin, 14 arm connecting pin, 15 bucket connecting pin, 18A, 18B, 18C stroke sensor, 31 engine, 33 rotation sensor, 34 hydraulic pump, 35 operation valve, 38 EPC valve, 50 main controller, 51 work equipment attitude acquisition unit, 52 excavation volume calculation unit, 53 work amount calculation unit, 58 work equipment control unit, 59 memory unit, 60 optimization controller, 100 hydraulic excavator, G ground.

Claims

1. a work machine having a bucket at a tip thereof and performing excavation work with the bucket; a controller for adjusting control parameters for controlling the excavation operation; The controller adjusting the control parameters using an evaluation function that is a function of the control parameters and includes a weight of a load to be loaded on the bucket during the excavation operation and a cost required for the excavation operation; A work machine control system that, in the process of adjusting the control parameters, determines the control parameters to be evaluated next by taking into consideration both the predicted average value of the evaluation function and the prediction uncertainty of the evaluation function.

2. The work machine control system according to claim 1 , wherein the cost includes an amount of work required for the excavation operation.

3. 2. The control system for a work machine according to claim 1, wherein the controller adjusts which of the multiple indexes included in the evaluation function is to be emphasized by weighting.

4. Further provided is a work machine actuator that drives the work machine, The work machine control system according to claim 1 , wherein the control parameters include command values ​​for operations of the work machine actuators during the excavation operation.

5. The work machine control system of claim 1 , wherein the control parameter includes an amount of change in angle of the bucket during the excavation operation.

6. 6. A work machine control system according to claim 1, wherein the controller estimates a safety area based on a safety index for the excavation work and a threshold value for the safety index, and determines the control parameter outside the safety area as the control parameter to be evaluated next, with the goal of expanding the safety area.

7. 7. The control system for a work machine according to claim 6, wherein the controller determines the control parameter that is within the safety region as the control parameter to be evaluated next, with the goal of maximizing or minimizing the evaluation function.

8. The work machine control system according to claim 6 , wherein the controller estimates the safety region based on the uncertainty of a prediction of the safety index and the threshold value of the safety index.

9. The control system for a work machine according to claim 6, wherein the controller uses one index of a plurality of indexes included in the evaluation function as the safety index.

10. calculating an evaluation function, which is a function of control parameters for controlling excavation work performed by a bucket at the tip of a work machine, and which includes a weight of a load to be loaded onto the bucket during the excavation work and a cost required for the excavation work; and adjusting the control parameters using the evaluation function; A control method for a work machine, wherein in the process of adjusting the control parameters, the control parameters to be evaluated next are determined taking into consideration both the predicted average value of the evaluation function and the prediction uncertainty of the evaluation function.

Images