Synthetic Plant Training Data Generation via 3D Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high cost and resource intensity of annotating training data for machine learning models to detect and classify plants, especially undesirable weeds, due to the scarcity of available images and the need for pixel-wise annotation.
Innovation Solution
Generating synthetic training images with automatically annotated 3D synthetic plants that reflect empirical evidence and agricultural data, including environmental conditions and time-series data, to train machine learning models for plant detection and classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-life digital images are used for training machine learning models, then model training can be performed on actual plant data, but the cost and resource requirements for annotating training data become prohibitively high
Solution Approach 1:
The patent creates synthetic copies of real plant images through 3D modeling and rendering. Instead of using actual photographs that require expensive pixel-wise annotation, the system generates synthetic images that replicate the visual characteristics, lighting conditions, and environmental contexts of real plant scenes. These synthetic copies serve as training data, eliminating the need for costly manual annotation while maintaining training effectiveness.
2Quantity of substance
If images of undesirable plants such as weeds are collected for training, then training data can be obtained, but such images are not as widely available or easily acquired as images of desirable plants
Solution Approach 1:
The patent segments the plant training data problem by creating separate 3D models for different plant types (desirable crops and undesirable weeds) with distinct visual characteristics. Each plant type can be independently modeled, textured, and instantiated in synthetic environments. This segmentation allows the system to generate balanced training datasets for both crop and weed detection, overcoming the scarcity of readily available weed images.
3Ease of manufacture
If synthetic training images are generated without empirical evidence, then generation cost is reduced, but the realism and effectiveness of the training images deteriorate
Solution Approach 1:
The patent systematically adjusts multiple parameters during synthetic image generation to enhance realism while controlling costs. This includes varying lighting conditions (sun angle, cloud cover, rain), environmental factors (soil texture, background vegetation), plant characteristics (leaf shapes, sizes, textures, heights), and temporal variations (growth stages, seasonal changes). By changing these parameters based on empirical agricultural data, the system generates diverse, realistic training images without requiring expensive manual creation.
Data Source
AI summary
Implementations are described herein for automatically generating synthetic training images that are usable as training data for training machine learning models to detect, segment, and/or classify various types of plants in digital images. In various implementations, a digital image may be obtained that captures an area. The digital image may depict the area under a lighting condition that existed in the area when a camera captured the digital image. Based at least in part on an agricultural history of the area, a plurality of three-dimensional synthetic plants may be generated. The synthetic training image may then be generated to depict the plurality of three-dimensional synthetic plants in the area. In some implementations, the generating may include graphically incorporating the plurality of three-dimensional synthetic plants with the digital image based on the lighting condition.