Asynchronous Retina Sensor Down Sampling via Integration Thresholding

Solution Overview

Problem

Current retinal prostheses face challenges in increasing pixel density, and standard computer vision techniques for down sampling asynchronous signals from retina sensors are processor-consuming and lose time information when converting signals to 2D frames.

Innovation Solution

A method for down sampling asynchronous signals from retina sensors by updating an integration value associated with a group of pixels and generating events when the value exceeds a threshold, allowing for direct processing in the asynchronous domain without converting to 2D, mimicking human neuron behavior and using weighting parameters to create recognition patterns.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If standard computer vision techniques are used to down sample asynchronous signals by converting to 2D frames, then down sampling can be performed, but processor consumption increases and time information is lost

Engineering Contradiction:

Improvedown sampling capabilityVSAvoidprocessor consumption

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The invention extracts and processes only the essential temporal information from asynchronous events, rather than converting entire frames to 2D images. By taking out only the necessary event data (coordinates and timestamps) and processing them directly in the asynchronous domain, the system achieves down sampling without the computational overhead of frame conversion, thus reducing processor consumption while maintaining time information integrity

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of following the conventional approach of converting asynchronous events to synchronous frames for down sampling, the invention inverts the process by performing down sampling directly on asynchronous events. This reversal eliminates the need for frame conversion entirely, addressing both the processor consumption issue and the time information loss problem simultaneously

Inventive Principle:

Principle #13The other way round (Inversion)

2Productivity

If standard computer vision techniques are used to down sample asynchronous signals by converting to 2D frames, then down sampling can be performed, but time information is lost

Engineering Contradiction:

Improvedown sampling capabilityVSAvoidtime information

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The invention extracts and processes only the essential temporal information from asynchronous events, rather than converting entire frames to 2D images. By taking out only the necessary event data (coordinates and timestamps) and processing them directly in the asynchronous domain, the system achieves down sampling without the computational overhead of frame conversion, thus reducing processor consumption while maintaining time information integrity

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of following the conventional approach of converting asynchronous events to synchronous frames for down sampling, the invention inverts the process by performing down sampling directly on asynchronous events. This reversal eliminates the need for frame conversion entirely, addressing both the processor consumption issue and the time information loss problem simultaneously

Inventive Principle:

Principle #13The other way round (Inversion)

3Manufacturing precision

If pixel density in retinal implants is increased, then visual resolution improves, but manufacturing complexity increases

Engineering Contradiction:

Improvevisual resolutionVSAvoidimplant complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The invention segments the high-resolution sensor output into lower-resolution regions that correspond to the implant's pixel density. By dividing the sensor matrix into groups of pixels that map to individual implant pixels, the system effectively adapts high-resolution asynchronous sensor data to lower-resolution implants without requiring the implant itself to have high pixel density, thus avoiding the manufacturing complexity of high-density implants while still achieving good visual resolution

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The invention changes the resolution parameter through software processing rather than hardware design. By implementing down sampling algorithms that aggregate multiple sensor pixels into single implant pixel activations, the system achieves effective resolution matching between sensor and implant without physically increasing implant pixel density, thereby avoiding the associated manufacturing complexity and cost

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS10500397B2Method for downsampling a signal outputted by an asynchronous sensor

Publication Date: 2019.12.10 SORBONNE UNIVERSITE

AI summary

Disclosed is a method for processing asynchronous signals generated by a light sensor, the sensor having a matrix of pixels, the method including: —receiving the asynchronous signals, each signal being associated with a pixel in a group of pixels in the matrix, each signal including successive events issued by the associated pixel; —upon an occurrence of an event in one of the asynchronous signals, updating an integration value associated to the group by adding an additive value to the integration value; —if the integration value is greater than a predetermined threshold, generating an event in an outputted asynchronous signal.