Rolling-Shutter Feature Alignment with Motion Estimation for Object Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Rolling-shutter cameras introduce distortions such as geometric, inconsistent exposure, temporal, motion blur, ambiguity in depth perception, and perspective distortions, affecting the accuracy of object detection and tracking, particularly in autonomous driving systems.

Innovation Solution

A method involving motion estimation and optimal transport for aligning image features, using techniques like optical flow and essential matrix decomposition to correct rolling-shutter distortions, warping distorted images to a single time instant, and generating corrected 3D point clouds for accurate object detection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If rolling-shutter camera is used for capturing images, then the camera can capture images sequentially row by row, but the image exhibits distortion when the camera or objects move during capture

Engineering Contradiction:

Improveimage capture speedVSAvoidimage geometric accuracy

Core Design Contradiction:

SpeedVSManufacturing precision

Solution Approach 1:

The system performs motion estimation and feature alignment before final object detection. By estimating motion parameters from aligned features and pre-adjusting the aligned features based on these parameters, the system compensates for rolling-shutter distortions before the detection process, thereby maintaining both capture speed and geometric accuracy

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent replaces mechanical/physical correction methods with computational approaches. Instead of using hardware to prevent distortion, the system uses image processing techniques including feature extraction, alignment algorithms, motion estimation, and feature adjustment to computationally correct rolling-shutter distortions after capture

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If feature alignment and motion estimation are performed to correct distortions, then object detection accuracy is improved, but computational complexity increases

Engineering Contradiction:

Improveobject detection accuracyVSAvoidprocessing algorithm complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system segments the correction process into distinct stages: generating image features from images, aligning features from multiple images, estimating motion parameters from aligned features, and adjusting features based on motion parameters. This segmentation allows each stage to be optimized independently and facilitates parallel processing, reducing overall computational complexity while maintaining detection accuracy

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS20250259283A1Alignment for improved perception based on image data

Publication Date: 2025.08.14 QUALCOMM INC

AI summary

Systems and techniques are described herein for detecting objects. For instance, a method for detecting objects is provided. The method may include generating first image features based on a first image; generating second image features based on a second image; aligning the first image features and the second image features to generate aligned features; estimating motion parameters based on the aligned features; adjusting the aligned features based on the motion parameters to generate adjusted aligned image features; and detecting an object based on the adjusted aligned image features.