Multi-Modal Compound Eye Perception for Degraded-Environment Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional visual perception methods struggle with low accuracy and robustness in complex degraded environments due to factors like limited viewing angles, lighting conditions, and weather changes, and existing deep learning-based methods face high computational complexity in multi-modal stitched images.
Innovation Solution
A multi-modal compound eye perception method using a trained feature point prediction model and nearest neighbor matching technique for image stitching, combined with a lightweight multi-modal perception detection network, to enhance target detection in complex environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional visual algorithms are used for target detection in complex degraded environments, then the system is simple to implement, but the detection accuracy and robustness are low due to limited viewing angles and environmental factors
Solution Approach 1:
The patent combines multiple imaging modalities (visible light, infrared, and other spectral bands) into a multi-modal compound eye perception system. This merging of different sensing modalities allows the system to overcome the limitations of single-modality detection in complex degraded environments, improving target detection accuracy while maintaining reasonable system complexity through integrated processing architecture
2Measurement precision
If deep learning-based target detection methods are applied to multi-modal stitched images, then the detection accuracy improves, but the computational complexity becomes too high for real-time application
Solution Approach 1:
The patent segments the target detection process into distinct stages: feature point extraction from multi-modal images, image stitching/alignment, and target detection. By dividing the complex deep learning task into modular components, the system can process multi-modal stitched images with improved accuracy while managing computational complexity at each stage to enable real-time application
3Ease of manufacture
If manual feature extraction methods like SIFT, SURF, or Harris corner detection are used, then the implementation is straightforward, but the methods are overly sensitive to scene changes including illumination, scale, and viewing angle variations
Solution Approach 1:
The patent replaces traditional manual feature extraction methods (SIFT, SURF, Harris) with deep learning-based feature point prediction models. This substitution eliminates the sensitivity to scene changes inherent in manual methods, significantly improving feature extraction robustness against illumination, scale, and viewing angle variations while maintaining implementation feasibility through trained neural network models
Data Source
AI summary
A multi-modal compound eye perception method and device for a complex degraded environment includes: acquiring multiple sets of images in the complex degraded environment through a multi-modal compound eye acquisition device, inputting them into a trained feature point prediction model to extract key feature point information of visible light images and infrared images; generating a visible light stitched image and an infrared stitched image based on a nearest neighbor matching technique, the key feature point information of visible light images and the infrared images, and inputting them into a constructed multi-modal perception detection network to perform target detection to obtain a multi-modal perception detection result.