Fisheye Object Recognition With Distortion-Aware CNN Kernels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Fisheye cameras provide distorted images, which hinder the use of existing artificial intelligence solutions for image recognition and scene understanding, and existing rectification techniques are limited by manual operations and hand-engineering, while direct CNN applications face challenges with radial distortion and computation needs.
Innovation Solution
A distortion-aware object recognition model that learns features from raw fisheye images without pre-processing, using a deep neural network with deformable kernels adapted to ultrawide field of view sensors, enabling efficient deployment on embedded systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image rectification techniques are used to correct distortion in fisheye images, then geometric accuracy is improved, but manual operations and pre-prepared configurations are required which reduce ease of operation
Solution Approach 1:
The system performs automatic self-calibration by detecting calibration patterns in the environment and computing distortion parameters without manual intervention. The calibration process is automated through machine learning models that identify patterns and calculate rectification parameters independently, eliminating the need for manual measurements and configurations.
Solution Approach 2:
The system pre-computes distortion correction parameters during an initial calibration phase using detected environmental patterns. These pre-computed parameters are then stored and applied during subsequent image processing operations, avoiding the need for repeated manual calibration while maintaining geometric accuracy.
2Adaptability or versatility
If deep convolutional neural networks are applied directly on fisheye images, then object recognition capability is improved, but radial distortion breaks down translation invariance property leading to inaccurate feature extraction
Solution Approach 1:
The system applies distortion correction as a preliminary step before feeding images to the convolutional neural network. By pre-processing the fisheye images to remove radial distortion, the translation invariance property of CNNs is preserved, enabling accurate feature extraction while maintaining the ability to recognize objects in various positions and orientations.
3Measurement precision
If geometric mapping from large FOVs of fisheye images to rectilinear space is performed, then distortion correction is achieved, but loss of FOV and scene information occurs
Solution Approach 1:
The system applies selective distortion correction only to regions of the image where objects are detected or where geometric accuracy is critical. Peripheral regions that contain less critical scene information retain their original fisheye characteristics, preserving the wide field of view and maximum scene coverage while correcting distortion locally where needed for accurate object recognition.
Data Source
AI summary
There is provided a method and system for training an object recognition machine learning model to perform object recognition in data acquired by ultrawide field of view (UW FOV) sensors to thereby obtain a distortion-aware object recognition model. The object recognition model comprises convolution layers each associated with a set of kernels. During training on a UW FOV labelled training dataset, deformable kernels are learned in a manifold space, mapped back to Euclidian space and used to perform convolutions to obtain output feature maps which are used to perform object recognition predictions. Model parameters of the distortion-aware object recognition model may be transferred to other architectures of object recognition models, which may be further compressed for deployment on embedded systems such as electronic devices on board autonomous vehicles.