Plant Image Labeling via RGB-Hyperspectral Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image classification algorithms in precision farming require hyperspectral image data, which are expensive and technically complex to acquire, while RGB images lack sufficient information content to replace hyperspectral images, leading to biased and inaccurate models due to the lack of large, manually annotated training datasets.
Innovation Solution
A method involving simultaneous or close succession image acquisition using cheaper RGB and hyperspectral techniques, followed by spatial alignment and machine learning to correlate features, allowing automatic label prediction in RGB images based on learned correlations with hyperspectral data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hyperspectral image data is used for image classification, then measurement precision and information content are improved, but device complexity and acquisition cost increase
Solution Approach 1:
The patent creates a computational copy of hyperspectral image data by training a machine learning model to generate synthetic hyperspectral images from RGB inputs. The model learns the mapping between RGB and hyperspectral spaces using paired training data, then produces pseudo-hyperspectral images that replicate the spectral information needed for accurate classification without requiring actual hyperspectral cameras during deployment
Solution Approach 2:
The patent replaces expensive, complex hyperspectral cameras with inexpensive RGB cameras for the actual image acquisition. The costly hyperspectral data is only used temporarily during the training phase to create the machine learning model, after which the system operates using only affordable RGB imagery
2Measurement precision
If manually annotated training datasets are created for machine learning, then model accuracy is improved, but productivity and time consumption worsen
Solution Approach 1:
The patent implements a self-labeling approach where the machine learning model automatically generates labels for training data by processing RGB images through the trained synthetic hyperspectral generation pipeline. The system performs automated semantic segmentation and class assignment without requiring manual human annotation, making the labeling process self-service and highly efficient
Solution Approach 2:
The patent performs preliminary generation of synthetic hyperspectral images and automatic labeling during the training phase. By pre-processing the training data with the trained model to generate labels and pseudo-hyperspectral images before final model deployment, the system eliminates the need for time-consuming manual annotation during operational phases
Data Source
AI summary
A computer-implemented method can include acquiring first training images using a first image acquisition technique, where each first training image depicts a plant-related motive; and acquiring second training images using a second image acquisition technique, where each second training image depicts the motive depicted in a respective one of the first training images. The method can include automatically assigning at least one label to each of the acquired second training images. The method can include spatially aligning the first and second training images which are depicting the same one of the motives into an aligned training image pair. The method can include training a machine-learning model as a function of the aligned training image pairs and the labels, wherein during the training the machine-learning model learns to automatically assign one or more labels to any test image acquired with the first image acquisition technique which depicts a plant-related motive.