Rolling Shutter Camera Motion Compensation via Vertical Plane Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Rolling shutter cameras in automotive applications suffer from significant distortion due to apparent camera motion, violating the perspective camera model and causing noticeable image distortions, especially in scenarios with quick camera movement.
Innovation Solution
A method for compensating camera motion in rolling shutter cameras by extracting line segments from images, fitting them to a simplified model of intersecting vertical planes, and using robust estimators like RANSAC to refine camera motion parameters, allowing for motion compensation and depth estimation from a single image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If rolling shutter camera is used for real-time image capture, then frame rate and response time are improved, but image distortion increases due to apparent camera motion
Solution Approach 1:
The patent changes the parameter representation by modeling the scene with simplified geometric primitives (vertical planes, horizontal planes) and their intersections (lines, points). This parameter transformation allows the system to work with fewer, more robust parameters that are less sensitive to rolling shutter distortion, enabling accurate reconstruction despite the distortion caused by high frame rate capture
Solution Approach 2:
The patent replaces traditional mechanical/optical solutions (such as using global shutter cameras or stabilizing gimbals) with a computational approach. By substituting the mechanical constraint of simultaneous row capture with an algorithmic correction method that processes rolling shutter images, the system achieves distortion compensation without sacrificing frame rate or requiring additional hardware
2Loss of time
If rolling shutter camera captures images during camera movement, then real-time processing is improved, but violation of perspective camera model occurs causing distortion
Solution Approach 1:
The patent applies preliminary action by pre-defining the geometric model of the scene (vertical planes, horizontal planes, and their intersections) before processing the rolling shutter image. This pre-established model framework allows the system to directly map distorted image features to the predefined geometric structure, enabling fast correction without requiring complex iterative optimization that would increase processing time
Solution Approach 2:
The patent transforms the problem from working with full 3D scene parameters to working with simplified parameters defined by plane intersections. By changing the parameter space to focus only on the essential geometric features (lines and points where vertical and horizontal planes intersect), the system maintains model reliability while achieving real-time processing speeds
3Measurement precision
If line segments are fitted to simplified model of vertical planes, then measurement precision is improved, but scene complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the complex scene into simpler geometric components - specifically, separating vertical planes from horizontal planes and focusing on their intersections. This segmentation allows the system to process line segments in a structured manner, fitting them to specific plane types, which improves measurement precision while keeping the overall model complexity manageable through modular processing
Solution Approach 2:
The patent applies local quality by assigning different geometric properties to different parts of the scene model. Vertical planes are modeled with specific constraints (extending infinitely in the vertical direction), while horizontal planes have different constraints. This local differentiation of geometric properties allows accurate camera motion estimation by exploiting the specific structural characteristics of each plane type, improving precision without requiring a uniformly complex model
Data Source
AI summary
A method of compensating for camera motion during capture of the image in a rolling shutter camera, the method comprising:receiving an image of a scene captured by a camera with a rolling shutter;extracting line segments in said image;estimating the movement of the camera during the capturing of the image from the received image; andproducing an image compensated for the movement during capture of the image,wherein the scene is approximated by a horizontal plane and two vertical planes that intersect at a line at infinity and estimating the movement of the camera during the capture of the image comprises assuming that the extracted line segments are vertical and lie on the vertical planes.