Robotic Stem Width Estimation Using Monocular Vision in Cluttered Fields
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current agricultural phenotyping methods are labor-intensive, inaccurate, and inefficient, particularly in field conditions with high clutter and varying environmental factors, limiting the ability to accurately measure crop stem width and population, which are crucial for optimizing yield and crop breeding.
Innovation Solution
A robotic system equipped with a side-facing monocular RGB camera, 2D LIDAR, and wheel encoders, utilizing image processing algorithms for foreground extraction and structure from motion to estimate stem width, and LIDAR for depth measurement, allowing for accurate and efficient phenotyping in real-field conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual measurements are used for plant phenotyping, then measurement accuracy can be maintained, but labor intensity and time consumption increase significantly
Solution Approach 1:
The patent replaces manual mechanical measurement systems with an automated robotic system that uses computer vision and image processing algorithms. The robot captures images of plant stems and automatically measures stem width through digital image analysis, eliminating the need for manual caliper measurements while maintaining accuracy and significantly increasing throughput.
Solution Approach 2:
The system enables self-service phenotyping by automating the entire measurement process. The robotic platform autonomously navigates through crop rows, captures images, processes them through foreground extraction algorithms, and generates phenotypic data without requiring continuous human intervention or manual measurement operations.
2Productivity
If automated phenotyping systems are deployed in field conditions, then productivity increases, but measurement accuracy decreases due to environmental clutter and varying conditions
Solution Approach 1:
The patent applies foreground extraction algorithms that separate the plant stem signal from the complex background clutter. The system extracts only the relevant foreground elements (stems) from images containing leaves, soil, and other field elements, enabling accurate measurements even in highly cluttered field conditions.
Solution Approach 2:
The system dynamically adapts to varying field conditions by using real-time image processing that adjusts to changing lighting, background clutter, and plant positions. The foreground extraction algorithms dynamically identify and track stems across varying environmental conditions, maintaining measurement accuracy throughout the field.
3Productivity
If comprehensive phenotyping data collection is implemented, then agricultural productivity optimization improves, but system complexity and cost increase
Solution Approach 1:
The robotic platform is designed as a universal phenotyping system that can measure multiple plant parameters (stem width, height, population density) across different crop types using the same core hardware and software architecture. This multi-functionality reduces overall system complexity compared to having separate specialized systems for each measurement type.
Solution Approach 2:
The system transitions from traditional 2D image capture to 3D phenotypic characterization by incorporating depth information and spatial context through the robotic platform's known position and orientation. This dimensional enhancement allows accurate stem width measurement without requiring complex multi-camera setups or specialized sensors.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The robotic system achieves high accuracy in stem width estimation, matching or exceeding manual measurements, and can be applied to various crops, providing a robust and efficient solution for phenotyping in challenging agricultural environments.
Implementation Method 1
obtain sensor data from a light detection and ranging (LIDAR) sensor, wherein the LIDAR sensor is attached to the ground mobile robot that is travelling along the lane defined by a row of crops, and wherein the sensor data includes at least a portion of the row of crops
Data Source
AI summary
Aspects of the subject disclosure may include, for example, obtaining video data from a single monocular camera, wherein the video data comprises a plurality of frames, wherein the camera is attached to a mobile robot that is travelling along a lane defined by a row of crops, wherein the row of crops comprises a first plant stem, and wherein the plurality of frames include a depiction of the first plant stem; obtaining robot velocity data from encoder(s), wherein the encoder(s) are attached to the robot; performing foreground extraction on each of the plurality of frames of the video data, wherein the foreground extraction results in a plurality of foreground images; and determining, based upon the plurality of foreground images and based upon the robot velocity data, an estimated width of the first plant stem. Additional embodiments are disclosed.