Video Compressive Sensing with Side Information for Image Reconstruction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Compressive sensing techniques for image acquisition often compromise image quality due to incorrect regularization during spatial and temporal information reconstruction, especially when using a single measurement with a time-dependent coded aperture.

Innovation Solution

Capturing a first image using a time-dependent aperture and a second image concurrently, with the processor reconstructing multiple images based on both, allowing for improved quality by reducing dependency on regularization and utilizing a shifting mask, rotating mask, or digital mirror device for modulation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If compressive sensing is used to reduce the number of measurements, then the data acquisition time and storage requirements are reduced, but the image quality deteriorates due to incorrect regularization during reconstruction

Engineering Contradiction:

Improvedata acquisition speedVSAvoidimage quality

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The patent divides the reconstruction process into two independent parts: spatial reconstruction (using compressive sensing measurements with regularization) and temporal reconstruction (using video prediction algorithms). This segmentation allows each part to be optimized separately, maintaining image quality while preserving the speed benefits of compressive sensing.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces side information from temporal predictions as an intermediary to assist the compressive sensing reconstruction. The predicted frames from previous and future time points serve as priors that guide the reconstruction process, reducing dependency on regularization parameters while improving reconstruction accuracy.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Device complexity

If a single measurement with time-dependent coded aperture is used, then the device complexity is reduced, but the reconstruction accuracy deteriorates due to incorrect regularization

Engineering Contradiction:

Improvedevice complexityVSAvoidreconstruction accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent performs preliminary temporal predictions using video prediction algorithms before executing the compressive sensing reconstruction. By predicting what the scene should look like at previous and future time points, the system prepares prior information that guides the reconstruction process, reducing the need for complex regularization in the actual reconstruction step.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent changes the approach from adjusting regularization parameters to adjusting temporal prediction parameters. Instead of tuning regularization strength to achieve good reconstruction, the system tunes video prediction algorithm parameters, which is more straightforward and leads to better reconstruction accuracy with simpler device configuration.

Inventive Principle:

Principle #35Parameter changes

3Manufacturing precision

If conventional compression techniques are used after acquiring raw data for each pixel, then the image quality is maintained, but the data acquisition time and processing complexity increase

Engineering Contradiction:

Improveimage qualityVSAvoidprocessing time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The patent applies dynamic temporal prediction that adapts to scene motion. The video prediction algorithm dynamically adjusts based on the motion characteristics of the scene, using optical flow or other motion estimation techniques to predict frame sequences, thereby maintaining quality while reducing the number of measurements needed compared to static compressive sensing.

Inventive Principle:

Principle #15Dynamics

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enhances image quality by capturing high-speed video at a low frame rate with reduced regularization dependency, achieving higher frame rates through joint reconstruction of coded measurements and side information.

Implementation Method 1

capturing a first image of a scene using light that has been modulated by a time-dependent aperture during an integration time interval

Methodology Applied

Scientific EffectLight modulation:

Implementation Method 2

receiving light from the scene at a beam splitter via an objective lens that is shared by the first camera and the second camera and the beam splitter provides a first portion of the received light to the first camera and a second portion of the received light to the second camera

Methodology Applied

Scientific EffectLight splitting:

Data Source

PatentUS10154204B2Video compressive sensing with side information

Publication Date: 2018.12.11 PIECE FUTURE PTE LTD

AI summary

A time-dependent aperture is configured to modulate light during an integration time interval. A first detector is configured to capture a first image of a scene during the integration time interval using light received from the time-dependent aperture. A second detector is configured to capture a second image of the scene concurrently with the first detector capturing the first image. A processor is configured to reconstruct a plurality of images of the scene based on the first image and the second image.