3D Plant Phenotyping With Point Cloud Completion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing plant phenotyping methods face challenges in accurately capturing and analyzing crop parameters due to limited viewing angles, environmental factors, and difficulties in differentiating between plant organs, leading to discrepancies and phenotyping errors.
Innovation Solution
A method and system using a mobile computing device with an image sensor and inertial sensor to capture images from multiple positions and orientations, generating point clouds, and supplementing distance coordinates to determine phenotypes with reliability metrics, enabling comprehensive phenotype measurements at various hierarchical levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single image sensor is used to capture crop fields, then the device complexity is low, but the measurement precision is insufficient due to limited viewing angle
Solution Approach 1:
The patent transitions from 2D image capture to 3D phenotyping by implementing a multi-sensor system that captures images from multiple positions and orientations. The sensor array captures data along a trajectory, enabling three-dimensional reconstruction of plant phenotypes through point cloud generation and spatial coordinate transformation, thereby achieving comprehensive 3D measurements without excessive complexity
Solution Approach 2:
The patent divides the imaging task into segments by using multiple sensors positioned at different locations. Each sensor captures a specific portion of the crop field from its own viewpoint, and the system integrates these segmented views through coordinate transformation and point cloud fusion to reconstruct complete plant phenotypes, resolving the contradiction between comprehensive measurement and system simplicity
2Measurement precision
If images are captured from multiple positions and orientations, then the phenotype measurement accuracy is improved, but the loss of time increases due to multiple capture positions
Solution Approach 1:
The patent implements continuous phenotyping data collection by moving the sensor array along a defined trajectory through the crop field. Rather than stopping at discrete positions, the system continuously captures images while in motion, with the processor simultaneously generating point clouds and transforming coordinates in real-time, thereby maintaining measurement precision while minimizing time loss through uninterrupted data acquisition
Solution Approach 2:
The patent performs preliminary spatial transformation by pre-establishing the relationship between sensor positions, orientations, and plant coordinate systems. The system pre-processes image data into point clouds and pre-calculates transformation matrices for coordinate conversion, enabling rapid phenotype extraction without time-consuming post-processing, thus reducing overall data collection time while maintaining accuracy
3Measurement precision
If point cloud data is used for phenotype determination, then the measurement precision is improved, but the device complexity increases due to additional processing requirements
Solution Approach 1:
The patent implements self-service processing where the sensor array and processor work as an integrated autonomous system. The sensors automatically capture images, the processor continuously generates point clouds and performs coordinate transformations without external intervention, and the system self-calibrates by tracking plant positions across multiple frames, thereby achieving high spatial measurement precision while managing processing complexity through automated self-sufficient operation
Solution Approach 2:
The patent replaces complex mechanical measurement systems with computational point cloud analysis. Instead of using physical measurement devices, the system uses image processing algorithms to generate point clouds and extract spatial coordinates mathematically, substituting mechanical complexity with computational efficiency while maintaining or improving measurement precision through digital reconstruction of plant phenotypes
4Measurement precision
If multiple plant organs are differentiated in dense crops, then the phenotype accuracy is improved, but the difficulty of detecting and measuring increases due to overlapping portions
Solution Approach 1:
The patent resolves organ differentiation difficulties by transitioning from 2D image analysis to 3D point cloud reconstruction. The multi-sensor system captures plant organs from multiple angles and reconstructs their three-dimensional positions and shapes, allowing clear differentiation of overlapping organs through spatial separation in 3D space, thereby achieving accurate organ-level phenotyping even in dense crop configurations where 2D views would be ambiguous
Data Source
AI summary
Method and system for plant phenotyping. Sequence of images of agricultural scene are captured along trajectory using image sensor of mobile computing device. Position and orientation data is obtained for each captured image using inertial sensor of mobile computing device. Point cloud generated in captured images, point cloud including coordinates defining distances to respective features in image. An object is selected, object belonging to plant hierarchy level of: plant organ; plant; plant grouping; or plant field. For each selected first object in first plant hierarchy level, object is identified and tracked in successive images of sequence, distance coordinates for object are supplemented when point cloud insufficient; visually undetectable portions of object are supplemented in images; spatial distance of object in respective images is determined based on point cloud coordinates and supplemented distance coordinates; and at least one phenotype of object is determined, based on determined spatial distance of object.