Rolling Shutter Sensor Extracts Information from Flicker Patterns
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies lack effective mechanisms for conveying information through displayed images, as they rely on human perception of modulated brightness, which is imperceptible to humans and requires complex signal processing to extract information from captured images.
Innovation Solution
The use of a rolling shutter sensor and hardware processor to capture and process two images, where information is represented as a flicker pattern superimposed on the display image, allowing for the separation of the information and display image through suitable signal processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If human perception of modulated brightness is used to convey information, then information can be conveyed through displayed images, but the modulated brightness is imperceptible to humans at high frequencies
Solution Approach 1:
The patent converts temporal modulation (time-based brightness changes) into spatial patterns (flicker patterns visible in captured images). By using a rolling shutter sensor that scans across the display over time, the temporal modulations are transformed into spatial flicker patterns that can be detected and decoded, effectively changing the dimension of information representation from temporal to spatial.
2Loss of information
If complex signal processing is used to extract information from captured images, then information can be recovered from modulated brightness, but the processing complexity increases
Solution Approach 1:
The rolling shutter sensor inherently performs the signal processing function through its scanning mechanism. As the sensor scans across the display during the modulation period, it automatically captures the temporal variations as spatial patterns without requiring additional external processing equipment. The sensor's readout architecture itself serves as the extraction mechanism, simplifying the overall system.
3Loss of information
If rolling shutter sensor is used to capture images, then temporally modulated light can be converted into spatial flicker pattern, but only two captured images are sufficient for extraction
Solution Approach 1:
The patent uses only two captured images with different exposure timings to extract both the display image and embedded information, rather than requiring continuous or excessive image capture. This partial action approach is sufficient because the rolling shutter sensor captures temporal information across the entire image during the exposure period, and two strategically timed captures provide enough data to separate the static display content from the modulated information signal.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables efficient communication between displays and cameras by converting temporally modulated light into a spatial flicker pattern, allowing for the extraction of embedded information from captured images, even when the display image is modulated at high frequencies imperceptible to humans.
Implementation Method 1
A message can be transmitted with these displays by temporally modulating a display's brightness at high frequencies (that are imperceptible to humans) while displaying an image. The message can then be captured by a rolling shutter camera which converts the temporally modulated incident light into a spatial flicker pattern.
Data Source
AI summary
Systems, methods, and media for extracting information and a display image from two captured images are provided, in some embodiments, systems for extracting information and a display image from two captured images are provided, the systems comprising: a rolling shutter sensor; and a hardware processor coupled to the rolling shutter sensor that is configured to: cause the rolling shutter sensor to capture two captured images; receive the two captured images; and extract the information, and the display image from the two captured images, wherein the information is represented in the captured images as a flicker pattern.