Fisheye Image Warping for Distortion-Reduced Object Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional image processing techniques struggle to effectively detect objects in images captured by fisheye lenses due to severe distortion, which complicates object detection in autonomous driving systems, as they cannot accurately recover the shape or position of objects in distorted fisheye images.

Innovation Solution

A system and method for fisheye image processing that partitions the fisheye image into smaller portions, applies specific transformations to each portion to reduce distortion, and stitches them back together, allowing for real-time processing and accurate object detection without compromising recovery quality across the entire image.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Area of stationary object

If fisheye lens is used to achieve wide field of view, then field of view is improved, but image distortion increases

Engineering Contradiction:

Improvefield of viewVSAvoidimage distortion

Core Design Contradiction:

Area of stationary objectVSShape

Solution Approach 1:

The fisheye image is divided into multiple smaller image portions or patches. Each portion is processed independently to remove distortion while preserving the wide field of view coverage. This segmentation allows distortion correction to be applied locally without compromising the overall wide-angle perspective.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different distortion correction transformations are applied to different portions of the image based on their local characteristics. Each image portion receives a customized transformation that optimizes distortion removal for that specific region, allowing for higher quality correction across the entire wide field of view.

Inventive Principle:

Principle #3Local quality

2Device complexity

If conventional image processing techniques are used on fisheye images, then processing simplicity is maintained, but object detection accuracy deteriorates

Engineering Contradiction:

Improveprocessing simplicityVSAvoidobject detection accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The image is segmented into multiple portions that can be processed using standard, simple transformation techniques. Each portion is corrected independently, allowing conventional processing methods to be applied effectively while achieving overall high detection accuracy across the entire image.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Distortion correction is applied as a preliminary step before object detection. By pre-processing the fisheye image to remove distortion and recover accurate object shapes and positions, subsequent object detection can proceed with standard techniques at high accuracy.

Inventive Principle:

Principle #10Preliminary action

3Shape

If traditional fisheye distortion reduction algorithms are used, then distortion removal is achieved, but adaptability to object detection applications deteriorates

Engineering Contradiction:

Improvedistortion removalVSAvoidadaptability to object detection

Core Design Contradiction:

ShapeVSAdaptability or versatility

Solution Approach 1:

The traditional distortion removal algorithm is adapted by dividing the image into multiple portions. Each portion can be processed with its own optimized transformation parameters, making the system adaptable to different object detection requirements while maintaining effective distortion removal.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The distortion correction transformations are made dynamic and adjustable for each image portion. This allows the system to adapt to different fisheye lens characteristics, viewing angles, and object detection needs, rather than applying a fixed transformation to the entire image.

Inventive Principle:

Principle #15Dynamics

4Reliability

If entire fisheye image is processed at once, then processing completeness is maintained, but processing speed deteriorates

Engineering Contradiction:

Improveprocessing completenessVSAvoidprocessing speed

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

Processing the image in segmented portions enables parallel processing of multiple regions simultaneously. This maintains completeness by ensuring all portions are processed, while significantly improving speed through parallel computation and reduced memory bandwidth requirements compared to processing the entire high-resolution image at once.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS12190465B2System and method for fisheye image processing

Publication Date: 2025.01.07 CREATEAI INC

AI summary

A system and method for fisheye image processing can be configured to: receive fisheye image data from at least one fisheye lens camera associated with an autonomous vehicle, the fisheye image data representing at least one fisheye image frame; partition the fisheye image frame into a plurality of image portions representing portions of the fisheye image frame; warp each of the plurality of image portions to map an arc of a camera projected view into a line corresponding to a mapped target view, the mapped target view being generally orthogonal to a line between a camera center and a center of the arc of the camera projected view; combine the plurality of warped image portions to form a combined resulting fisheye image data set representing recovered or distortion-reduced fisheye image data corresponding to the fisheye image frame; generate auto-calibration data representing a correspondence between pixels in the at least one fisheye image frame and corresponding pixels in the combined resulting fisheye image data set; and provide the combined resulting fisheye image data set as an output for other autonomous vehicle subsystems.