Ground Engaging Tool Tracking for Occlusion-Aware Loss Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting missing or worn ground engaging tools (GETs) on machines, such as excavator teeth, are inadequate as they do not accurately model the relationship between tools on a bucket, leading to potential damage and inefficiencies.
Innovation Solution
A method using image processing and neural networks to generate segmentation masks, calculate centroids, and determine distances between GETs, with notifications for missing tools, and occlusion detection based on depth metrics and similarity scoring.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image processing is used to detect missing ground engaging tools, then detection accuracy is improved, but false detections occur when tools are temporarily occluded
Solution Approach 1:
The system dynamically tracks GETs across multiple video frames rather than making static detections. By maintaining temporal continuity of GET positions and using motion consistency checks, the system distinguishes between temporarily occluded tools (which maintain their tracked position) and actually missing tools (where tracking is lost), thereby reducing false detections while maintaining high detection accuracy
Solution Approach 2:
The system establishes expected GET positions and relationships before analysis by creating a digital twin model of the bucket with all GETs in their proper locations. This preliminary modeling allows the system to compare actual detections against expected positions and identify discrepancies, improving detection accuracy while using contextual information to filter false positives from temporary occlusions
2Measurement precision
If the system monitors all ground engaging tools continuously, then loss detection accuracy is improved, but computational complexity and processing time increase
Solution Approach 1:
The system segments the monitoring task by creating a digital twin model that represents the expected state of all GETs on the bucket. Instead of processing each GET independently in real-time, the system divides the problem into model creation (offline) and comparison (online) phases, reducing real-time computational complexity while maintaining high detection accuracy through the segmented approach
Solution Approach 2:
The system creates a digital twin (copy) of the bucket and all its GETs in proper positions. This copy serves as a reference model that can be compared against actual camera detections. By using this pre-established copy, the system avoids the complexity of real-time reconstruction and relationship modeling, simplifying the detection process while maintaining accurate loss detection
3Reliability
If distance-based validation is used to identify missing tools, then detection reliability is improved, but the system fails when tools are temporarily occluded
Solution Approach 1:
The system maintains continuous tracking of each GET across video frames, establishing temporal continuity of their positions and motions. When a GET appears missing based on distance validation, the system checks whether this is a temporary occlusion by examining the continuity of tracking data from previous frames and expected motion patterns, thereby reducing false positives while maintaining detection reliability
Solution Approach 2:
The system uses feedback from multiple sources including tracking consistency across frames, expected motion patterns, and contextual information from neighboring GETs to validate distance-based missing tool detections. This multi-layered feedback mechanism allows the system to distinguish between temporary occlusions (where feedback indicates the tool should still be present) and actual losses (where feedback confirms the tool is genuinely missing), improving reliability while reducing false positives
Data Source
AI summary
A technique is directed to methods and systems for ground engaging tool (GET) loss detection. A GET monitoring system can receive image data from one or more cameras mounted on a machine. The image data can include image frames that include one or more GETs, such as bucket tips, lip shrouds, and/or side shrouds. The GET monitoring system can utilize a neural network to determine whether a GET is missing based on comparing consecutive images, depth measurements from the camera, or the distance relationships between multiple GETs. If a GET is determined to be missing from a bucket, the GET monitoring system can generate a notification to alert the operator or personnel at the site.