Spherical CNN for Wide-Angle Camera Image Distortion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Wide-angle cameras produce fisheye distortion, which affects the accuracy of deep learning and CNNs in environment perception and recognition, leading to degraded classification results due to varying distortions based on distance from the camera center.
Innovation Solution
Applying a convolutional neural network (CNN) to wide-angle camera images by mapping them onto a 3D surface, specifically using spherical harmonics for convolution, which maintains linear and shift invariance, and using intrinsic camera calibration values to eliminate distortion without dividing the image into multiple regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If wide-angle camera is used to increase field-of-view, then captured area is improved, but image distortion worsens
Solution Approach 1:
The patent maps the distorted wide-angle image onto a spherical surface to compensate for the fisheye distortion. By representing the image on a curved spherical manifold rather than a flat plane, the geometric relationships and object shapes are preserved more accurately across the entire field-of-view, resolving the shape distortion issue while maintaining the wide angular coverage
2Productivity
If standard CNN is applied to distorted wide-angle images, then processing speed is maintained, but classification accuracy worsens
Solution Approach 1:
The patent transforms the 2D distorted image into a 3D spherical representation, adding a dimensional aspect that accounts for the curved geometry of wide-angle images. This spherical CNN architecture operates on the manifold of the sphere, allowing standard convolutional operations to be applied in a geometrically appropriate space, thereby improving classification accuracy without sacrificing processing efficiency
3Shape
If image is divided into multiple regions to correct distortion, then local shape accuracy is improved, but device complexity worsens
Solution Approach 1:
The patent employs a universal spherical CNN architecture that handles the entire wide-angle image as a single coherent spherical manifold rather than dividing it into multiple regions. This unified approach maintains local shape accuracy across the whole field-of-view while avoiding the computational complexity and discontinuities associated with partitioning the image into separate regions for processing
Data Source
AI summary
An apparatus for applying a convolutional neural network (CNN) to a wide-angle camera image is described herein. The apparatus includes a camera, controller, convolution mechanism and a fully connected layer. The camera is to capture a wide-angle image, and the controller is to map the image on a 3D surface. The convolution mechanism is to perform convolution on the 3D surface and the fully connected layer is to classify a plurality of features generated by the convolution mechanism.