Hyperspectral Image Segmentation With VNIR-SWIR Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional plant phenotyping methods are labor-intensive, time-consuming, and prone to human error, and existing image segmentation techniques struggle with the complexity and variability of plant structures, lighting conditions, and multispectral data challenges in hyperspectral imaging, hindering accurate extraction of phenotypic information.
Innovation Solution
A deep learning network (DNN) model that processes VNIR and SWIR images through dual-input channels, utilizing spatial and channel attention mechanisms, edge detection, and a fusion layer to generate precise binary segmentation maps, optimized by SCA-ACO hyperparameter tuning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hyperspectral imaging is used to capture detailed plant physiological information, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent segments the hyperspectral image processing task into multiple stages: initial segmentation to separate plant from background, followed by detailed phenotypic trait extraction. This segmentation approach breaks down the complex high-dimensional data into manageable components, reducing processing complexity while maintaining measurement precision for detecting subtle plant tissue differences
Solution Approach 2:
The patent introduces an intermediary segmentation mask that separates the plant region from the background before performing detailed phenotypic analysis. This intermediary step simplifies the subsequent analysis by focusing computational resources only on the relevant plant regions, thereby reducing the effective complexity of hyperspectral data processing while preserving measurement precision
2Ease of operation
If standard segmentation techniques are used for simplicity, then ease of operation is improved, but manufacturing precision deteriorates
Solution Approach 1:
The patent employs dynamic segmentation that adapts to varying plant structures, lighting conditions, and growth stages. The segmentation approach adjusts its parameters and complexity based on the specific characteristics of the plant being analyzed, maintaining ease of operation through automated adaptation while achieving high precision in delineating plant structures and overlapping leaves
3Measurement precision
If dual-input channel DNN model is used to process VNIR and SWIR images, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent merges VNIR and SWIR image processing into a unified dual-input DNN model that shares common architectural components and processing pathways. By combining the processing of both spectral bands in a single integrated model with shared layers and parameters, the patent reduces the overall complexity compared to using separate independent models, while maintaining high segmentation accuracy through the complementary information from both spectral ranges
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed is a computer-implemented method for performing image segmentation of hyperspectral images, that includes capturing a VNIR image of an object using a VNIR image sensor, capturing a SWIR image of an object using a SWIR image sensor, and providing a deep learning network (DNN) model that is configured to receive the captured VNIR and SWIR images independently through two inputs of a dual-input channel, encode the VNIR and SWIR images to perform feature extraction and generate corresponding VNIR and SWIR feature maps, combine the VNIR and SWIR feature maps through a fusion layer, to generate a fusion feature map, wherein the fusion feature map is generated by performing element-wise addition of the VNIR and SWIR feature maps, and decode the fusion feature map through a single decoding path that includes a series of deconvolution blocks, and generate a binary segmentation map by an output layer based on the decoded feature map for delineating foreground from background of the object.