Grade Control Using Visual Marker Tracking

Solution Overview

Problem

Existing work machine grade control systems lack a simplistic external vertical reference for gauging the height of the cut, necessitating costly equipment or complex setups like laser planes and GPS, which is not always feasible.

Innovation Solution

A work machine equipped with a sensor that captures images, labels visual markers, and uses them to track the position of a movable grading element, adjusting its pitch, yaw, and roll to achieve a desired grade without external references, utilizing an inertial measurement unit for gravitational reference and stitching together keyframes for 3D mapping.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If costly equipment such as laser planes and ultrasonic sensors are added to provide external vertical reference, then measurement precision is improved, but device complexity and cost increase

Engineering Contradiction:

Improvevertical reference measurementVSAvoidequipment complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system uses the work machine's own inertial measurement unit and onboard camera to capture images and track visual markers, eliminating the need for external reference equipment. The machine serves itself by using its own sensors and processing capabilities to achieve precise vertical reference measurement

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces mechanical/optical reference systems (laser planes, ultrasonic sensors) with a vision-based system using cameras and image processing. The system captures images, identifies visual markers, and calculates position through software algorithms rather than physical reference equipment

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If GPS connection is added to provide external reference, then measurement precision is improved, but device complexity and setup requirements increase

Engineering Contradiction:

Improveposition referenceVSAvoidsystem setup complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system uses the work machine's own inertial measurement unit and onboard camera to capture images and track visual markers, eliminating the need for external reference equipment. The machine serves itself by using its own sensors and processing capabilities to achieve precise vertical reference measurement

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent introduces visual markers as an intermediary element that can be easily placed in the work area. These markers serve as reference points that the camera system tracks to determine position and orientation, replacing complex GPS infrastructure with simple visual cues

Inventive Principle:

Principle #24Intermediary (Mediator)

3Manufacturing precision

If constant adjustments to blade position are made to achieve slope, angle, and elevation targets, then manufacturing precision is improved, but productivity decreases due to slow and steady operation required

Engineering Contradiction:

Improvegrade precisionVSAvoidgrading speed

Core Design Contradiction:

Manufacturing precisionVSProductivity

Solution Approach 1:

The system continuously captures images, tracks visual markers, compares actual position with target grade, and provides real-time feedback to the operator through the display. This closed-loop feedback enables precise grade control while allowing operators to work at normal speeds rather than requiring slow, methodical adjustments

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent replaces mechanical/optical reference systems (laser planes, ultrasonic sensors) with a vision-based system using cameras and image processing. The system captures images, identifies visual markers, and calculates position through software algorithms rather than physical reference equipment

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enables precise and efficient grading without additional costly equipment, providing a 3D model of the worksite and allowing for accurate adjustment of the grading element based on visual marker tracking, thus improving grading precision and productivity.

Implementation Method 1

An inertial measurement unit may be coupled to the work machine wherein the inertial measurement unit provides a gravitation reference

Methodology Applied

Scientific EffectGravitation: Gravitation

Data Source

PatentUS12173469B2Apparatus and method for grade control

Publication Date: 2024.12.24 DEERE & CO

AI summary

A work machine including a frame, a ground-engaging element movably coupled to the frame, a movable grading element, an actuator coupled to the movable grading element to controllably drive movement of the movable grading element engaging material to be graded, a sensor, and a controller. The controller comprises a memory that stores computer-executable instruction and a processor that executes the instructions. The instructions include labelling each of at least a plurality of pixels in a first image as a visual marker; selecting the first image as a reference keyframe; tracking at least one region of a subsequent image including the visual marker relative to the reference keyframe to determine an estimate of the current pose as the work machine moves; and adjusting a position of the movable grading element.