Fisheye Image Projection for Real-Time Autonomous Feature Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional autonomous driving systems face challenges in accurately detecting features from wide field of view sensors due to distortion, requiring significant computational resources and manual effort for retraining neural networks, which limits scalability and adaptability.
Innovation Solution
The method involves stereographically projecting images from wide field of view sensors onto a 2D plane, allowing existing neural networks trained on narrower field of view sensors to detect features in real-time, without the need for retraining, by virtually adjusting the field of view and using existing DNNs to classify features in distorted regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional DNNs are retrained using image data from wide field of view sensors, then feature detection accuracy for wide sensors is improved, but computational cost and manual effort increase significantly
Solution Approach 1:
The patent creates a virtual copy of the narrow FOV sensor data by projecting wide FOV images onto a 2D plane to simulate the appearance and characteristics of narrow FOV images. This allows the existing DNN trained on narrow FOV data to process wide FOV sensor inputs without retraining, effectively copying the processing pipeline while adapting the input representation.
Solution Approach 2:
The patent transforms the input data parameters by applying stereographic projection to convert wide FOV images into a 2D projected image space. This parameter transformation changes the geometric representation of features while preserving their detectability, allowing the DNN to operate on transformed data that matches its training distribution.
2Adaptability or versatility
If multiple instances of DNN are trained for different fields of view, then adaptability to various sensors is improved, but device complexity increases
Solution Approach 1:
The patent makes a single DNN instance universal by enabling it to process both narrow FOV and wide FOV sensor data through the projection mechanism. The same trained model can handle multiple sensor types and FOV configurations, eliminating the need for multiple specialized DNN instances and reducing system complexity.
Solution Approach 2:
The stereographic projection process acts as an intermediary between wide FOV sensors and the narrow FOV-trained DNN. This intermediate transformation layer adapts the wide FOV input data into a format compatible with the existing DNN, serving as a mediator that enables sensor diversity without requiring multiple models.
3Area of stationary object
If wide field of view sensors are used, then coverage area is improved, but image distortion increases
Solution Approach 1:
The patent addresses distortion by transforming the 2D distorted wide FOV image into a different 2D representation through stereographic projection. This dimensionality transformation maps the distorted features onto a projected plane where they regain geometric accuracy while preserving the wide field of view coverage.
Data Source
AI summary
In various examples, live perception from wide-view sensors may be leveraged to detect features in an environment of a vehicle. Sensor data generated by the sensors may be adjusted to represent a virtual field of view different from an actual field of view of the sensor, and the sensor data—with or without virtual adjustment—may be applied to a stereographic projection algorithm to generate a projected image. The projected image may then be applied to a machine learning model—such as a deep neural network (DNN)—to detect and/or classify features or objects represented therein. In some examples, the machine learning model may be pre-trained on training sensor data generated by a sensor having a field of view less than the wide-view sensor such that the virtual adjustment and/or projection algorithm may update the sensor data to be suitable for accurate processing by the pre-trained machine learning model.