Excavator Bucket Control With RL Depth Constraints

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing excavation methods lack adherence to geometric constraints, leading to unnecessary bucket movements and reworking of excavated areas, which are suboptimal in terms of efficiency and productivity.

Innovation Solution

A computer-implemented method using reinforcement learning to generate maximum-depth constraints, training a control model to infer control data that adhere to these constraints, thereby optimizing excavator operations and reducing unnecessary digging.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If traditional excavation automation methods are used, then the excavation task can be completed, but unnecessary bucket movements and reworking occur due to lack of adherence to geometric constraints

Engineering Contradiction:

Improveadherence to geometric excavation constraintsVSAvoidexcavation efficiency

Core Design Contradiction:

Manufacturing precisionVSProductivity

Solution Approach 1:

The system performs preliminary action by pre-calculating maximum-depth constraints based on the desired excavation shape before the actual excavation begins. The constraint generator creates depth constraints that guide the bucket movements, ensuring that the excavation stays within geometric boundaries from the start, thereby eliminating the need for reworking.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback by continuously monitoring the bucket state and terrain perception during excavation, comparing actual progress against the pre-calculated maximum-depth constraints. This real-time feedback allows the control system to adjust bucket movements dynamically, ensuring adherence to geometric constraints while maintaining efficient productivity.

Inventive Principle:

Principle #23Feedback

2Extent of automation

If traditional state-machine coordination is used, then the excavation process can be automated, but unnecessary bucket movements occur due to suboptimal control decisions

Engineering Contradiction:

Improveautonomous excavation capabilityVSAvoidtime for reworking excavated areas

Core Design Contradiction:

Extent of automationVSLoss of time

Solution Approach 1:

The system replaces traditional mechanical state-machine coordination with an intelligent control system based on reinforcement learning. The trained control model processes terrain perception and bucket state information to generate optimal control decisions, substituting rigid state-machine logic with adaptive learning-based control that minimizes unnecessary movements and reworking time.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Manufacturing precision

If maximum-depth constraints are enforced, then reworking is avoided, but the control problem becomes more complex

Engineering Contradiction:

Improveexcavation depth accuracyVSAvoidcontrol system complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The system reduces control complexity by performing preliminary calculation of maximum-depth constraints based on the desired excavation shape. By pre-computing these constraints before excavation begins, the system transforms a complex real-time optimization problem into a simpler tracking problem where the control system only needs to follow pre-determined depth boundaries.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system changes parameters by transforming the complex multi-dimensional excavation control problem into a simplified constraint-following problem. The reinforcement learning model is trained with maximum-depth constraints as key parameters, allowing the complex control task to be solved through learned policies that naturally adhere to depth boundaries without requiring complex real-time optimization.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentEP4653626A1Autonomous excavation based on a control policy trained with reinforcement learning under depth constraints

Publication Date: 2025.11.26 GRAVIS ROBOTICS AG

Figure 1A
Figure 1B~2
Figure 3

AI summary

The invention is notably directed to a computer-implemented method of controlling a bucket (110) of an excavator (1) for autonomous excavation, wherein the method comprises accessing (S5) a specification of a shape to be excavated in a terrain (150), and executing (S30) a control policy (206) including a constraint generator (2061) and a control model (2062) trained with reinforcement learning (preferably in simulation), for the control policy (206) to repeatedly perform (S30) algorithmic cycles. Each cycle of the algorithmic cycles comprises: updating (S32, S33) a state of the bucket (110) and a perception of the terrain (150) based on signals received from a set of sensors (410, 430), which preferably are on board the excavator (1); based on the accessed specification, the updated state of the bucket (110), and the updated perception, generating (S34) one or more maximum-depth constraints (164) through the constraint generator (2061), and inferring (S35), through the control model (2062), control data meeting the one or more maximum-depth constraints (164) generated; and instructing (S38) to control the bucket (110) based on actuator control signals formed (S36) according to the control data inferred, to excavate (S40) the terrain (150) in accordance with said specification. The invention is further directed to related systems and computer program products.