Fisheye Image Projection for Real-Time Autonomous Feature Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems require significant computational resources and manual effort to retrain deep neural networks (DNNs) for detecting features in images from wide field of view sensors, which are not scalable and accurate due to distortion and imperfections in image data, limiting their reliability in autonomous driving applications.
Innovation Solution
The method involves stereographically projecting images from wide field of view sensors onto a 2D plane, allowing existing DNNs trained on narrower field of view images to detect features in real-time, without the need for retraining, by virtually adjusting the field of view and using existing neural networks to classify features in distorted regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional systems retrain DNNs for wide field of view sensors, then feature detection accuracy may improve, but computational cost and manual effort increase significantly
Solution Approach 1:
The patent creates a virtual copy of the narrow FOV image by projecting the wide FOV image through a virtual pinhole camera model. This virtual copy has the same geometric properties as narrow FOV images, allowing the pre-trained DNN to process it without retraining. The projection transforms coordinates and applies distortion correction to generate an synthetic narrow FOV representation from the wide FOV input.
Solution Approach 2:
The patent changes the field of view parameter by virtually adjusting it from the actual wide FOV to an equivalent narrow FOV. This is achieved by modifying the projection parameters (focal length, principal point) in the pinhole camera model to match those of a narrow FOV sensor, thereby transforming the image geometry to be compatible with pre-trained models without retraining.
2Adaptability or versatility
If multiple DNN instances are trained for different fields of view, then adaptability to various sensors improves, but system complexity increases
Solution Approach 1:
The patent makes a single DNN instance universal by enabling it to process both narrow and wide FOV images through the virtual projection approach. The same pre-trained model can handle any FOV by transforming the wide FOV image to virtual narrow FOV coordinates, eliminating the need for multiple specialized models and simplifying the system architecture.
Solution Approach 2:
The patent introduces a virtual projection module as an intermediary between the wide FOV image sensor and the pre-trained DNN. This mediator transforms the wide FOV image into a virtual narrow FOV representation that the DNN can process, serving as a bridge that enables compatibility without requiring model retraining or multiple instances.
3Area of stationary object
If wide field of view images are used directly, then coverage area increases, but distortion and artifacts increase
Solution Approach 1:
The patent changes the distortion parameters by applying a virtual pinhole projection that corrects the wide FOV distortion. By transforming the image through a model with different focal length and principal point parameters, the distorted wide FOV image is converted to a less distorted virtual narrow FOV image, improving geometric accuracy while preserving the original wide area coverage capability.
Data Source
AI summary
In various examples, sensor data may be adjusted to represent a virtual field of view different from an actual field of view of the sensor, and the sensor data—with or without virtual adjustment—may be applied to a stereographic projection algorithm to generate a projected image. The projected image may then be applied to a machine learning model—such as a deep neural network (DNN)—to detect and/or classify features or objects represented therein.