Simulated Rolling Shutter Image Data Generation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Global shutter image data cannot be directly used by systems designed for rolling shutter image data due to differences in image capture methods, leading to distortions and artifacts, which hinders the use of rolling shutter images in applications like autonomous vehicle navigation.

Innovation Solution

Converting global shutter image data into simulated rolling shutter image data by determining rolling shutter artifacts and generating simulated images with optical flow fields, scan line characteristics, and high-dynamic range (HDR) simulations to create data compatible with rolling shutter image systems.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If global shutter image data is used directly in systems designed for rolling shutter data, then the system can maintain its original design and operation, but the image data cannot be properly processed due to fundamental differences in capture methods leading to distortions and artifacts

Engineering Contradiction:

Improvecompatibility between global shutter data and rolling shutter systemsVSAvoidimage quality and accuracy

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent creates simulated rolling shutter image data by copying and transforming global shutter image data through a virtual rolling shutter model. This involves generating synthetic rolling shutter artifacts (skew, distortion, timing variations) based on the optical flow field and scene geometry, thereby producing data that mimics真实 rolling shutter characteristics without requiring actual rolling shutter capture

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent transforms global shutter image data into simulated rolling shutter data by changing key parameters including temporal sampling characteristics, spatial distortion parameters, and scan line timing parameters. This parameter transformation enables the data to conform to the expected input characteristics of rolling shutter-based systems

Inventive Principle:

Principle #35Parameter changes

2Speed

If rolling shutter image capture method is used to record rapidly moving objects or flashing light, then the imager can operate with simpler hardware and faster readout, but the recorded images contain distortions and artifacts that differ from global shutter images

Engineering Contradiction:

Improvereadout speed and frame rateVSAvoidimage accuracy and distortion-free capture

Core Design Contradiction:

SpeedVSManufacturing precision

Solution Approach 1:

The patent converts the harmful rolling shutter artifacts (skew, distortion, temporal mismatches) into beneficial training data characteristics. By intentionally introducing these artifacts into simulated rolling shutter images derived from accurate global shutter data, the system creates realistic training examples that help machine learning models learn to handle and correct such distortions in real-world scenarios

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Solution Approach 2:

The patent performs preliminary simulation of rolling shutter effects on global shutter image data before the data is used for training or processing. This advance transformation prepares the data in advance to match the expected input characteristics of rolling shutter systems, preventing potential processing errors and improving system performance

Inventive Principle:

Principle #10Preliminary action

3Reliability

If a system is designed specifically for global shutter image data, then it can process that data accurately, but it cannot effectively use rolling shutter image data without modification

Engineering Contradiction:

Improveprocessing accuracy for global shutter dataVSAvoidability to process different shutter types

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent creates a universal data representation that can serve both global shutter and rolling shutter systems. By generating simulated rolling shutter data from global shutter inputs, the system enables a single processing pipeline to handle multiple shutter types effectively, making the system multi-functional and adaptable to different imaging modalities

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11483480B2Simulated rolling shutter image data

Publication Date: 2022.10.25 ZOOX INC

AI summary

A system for converting global shutter image data into simulated rolling shutter image data may include a global-to-rolling shutter image converter. The converter may be configured to receive global shutter image data associated with a plurality of global shutter images. The global shutter image data may include dynamic global shutter image data representative of a moving object in the plurality of global shutter images. The converter may also be configured to determine an optical flow field based at least in part on two or more of the plurality of global shutter images and generate the simulated rolling shutter image based at least in part on the optical flow field. The converter may also be configured to generate high dynamic range (HDR) images based at least in part on the optical flow field and the global shutter image data.