Asynchronous Event-Based Optical Flow Estimation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for estimating optical flow are slow and not well-suited for real-time applications, particularly with conventional cameras, and there is a lack of practical methods for using event-based vision sensors to determine optical flow effectively.

Innovation Solution

A method that utilizes asynchronous information from event-based vision sensors to select and quantify the variations in event times across a pixel matrix, employing a space-time representation to estimate velocity fields with high precision and speed, including the use of slope estimation and smoothing operations to attenuate noise.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional optical flow methods (correlation-based or gradient-based) are used, then motion detection capability is achieved, but execution speed is slow and real-time processing is difficult

Engineering Contradiction:

Improvemotion detection accuracyVSAvoidexecution speed

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent replaces conventional frame-based image processing with event-based asynchronous processing. Instead of processing complete image frames sequentially, the system processes individual pixel events as they occur, substituting the mechanical frame-by-frame processing approach with an event-driven paradigm that achieves both real-time performance and motion detection accuracy

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent segments the image processing task into independent pixel-level events. Each pixel operates independently to detect and report intensity changes, allowing parallel processing of multiple spatial locations simultaneously. This segmentation enables the system to achieve high execution speed while maintaining measurement precision through distributed event processing

Inventive Principle:

Principle #1Segmentation

2Productivity

If EMD (Elementary Motion Detector) is used for motion detection, then real-time processing is improved, but measurement precision deteriorates due to sensitivity to image contrast

Engineering Contradiction:

Improvereal-time processing capabilityVSAvoidvisual motion measurement accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent implements dynamic thresholding where the sensitivity threshold for event detection is adapted based on local image characteristics and historical event rates. This dynamic adjustment allows the system to maintain high real-time processing capability while compensating for varying image contrast conditions, thereby preserving measurement precision across different lighting and contrast scenarios

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system incorporates feedback mechanisms where detected events and their temporal patterns are used to adjust detection parameters and filter settings in real-time. This feedback loop enables the system to maintain measurement precision by adapting to changing scene statistics while preserving the fast response characteristics of event-based processing

Inventive Principle:

Principle #23Feedback

3Loss of information

If conventional cameras are used to record successive images, then complete image information is captured, but latency time increases and redundancy is high

Engineering Contradiction:

Improveimage information completenessVSAvoidlatency time

Core Design Contradiction:

Loss of informationVSLoss of time

Solution Approach 1:

The patent extracts only the essential motion-related information from the visual scene by detecting intensity changes at individual pixels. Instead of capturing and processing complete image frames, the system extracts and processes only the change events, eliminating redundant static information while preserving all motion-related data. This extraction approach reduces latency and information loss simultaneously

Inventive Principle:

Principle #2Taking out (Extraction)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach enables fast and accurate estimation of optical flow, reducing latency and increasing dynamic range, making it suitable for applications requiring fine motion control and real-time processing.

Implementation Method 1

Each pixel of the sensor is independent and detects changes in intensity above a threshold since the emission of the last event

Methodology Applied

Scientific EffectPhotoelectric Effect: Photoelectric Effect

Data Source

PatentUS9213902B2Method of estimating optical flow on the basis of an asynchronous light sensor

Publication Date: 2015.12.15 CENT NAT DE LA RECH SCI (C N R S)

AI summary

A computer receives asynchronous information originating from a light sensor (10) having a pixel matrix disposed opposite a scene. The asynchronous information comprises, for each pixel of the matrix, successive events originating from this pixel and depending on variations in light in the scene. For a place of estimation (p) in the matrix of pixels and an estimation time (t), the computer selects a set (Sp t) of events originating from pixels included in a spatial neighborhood (πρ) of the place of estimation and which have occurred in a time interval (Θ) defined with respect to the estimation time, such that this set has at most one event per pixel of the spatial neighborhood. The computer quantifies the variations in the times of occurrence of the events of the set selected as a function of the positions, in the matrix, of the pixels from which these events originate.