Event Vision Depth Sensor Sub-Pixel Positioning

Solution Overview

Problem

Conventional depth estimation techniques face limitations in resolution, accuracy, and latency due to the need for storing intensity values from all camera pixels for each illumination pattern, leading to high memory requirements and prolonged processing times.

Innovation Solution

A depth sensor device and method utilizing event vision sensors (EVS)/dynamic vision sensors (DVS) that detect changes in intensity, generating events at pixels with significant changes, allowing for sub-pixel accurate calculation of illumination pattern positions without storing full intensity information, thereby reducing memory usage and increasing processing speed.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If full intensity values from all camera pixels are stored for each illumination pattern, then complete intensity information is available for depth calculation, but memory requirements increase and processing time increases

Engineering Contradiction:

Improvedepth determination accuracyVSAvoidmemory space

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent extracts only the essential information needed for depth calculation - specifically, the position of the illumination pattern image on the pixel array - rather than storing complete intensity values from all pixels. This is achieved by detecting the center position of the illumination pattern through moment calculations or correlation methods, thereby eliminating the need to store redundant intensity data while preserving the critical positional information required for accurate depth determination.

Inventive Principle:

Principle #2Taking out (Extraction)

2Measurement precision

If full intensity values from all camera pixels are stored for each illumination pattern, then complete intensity information is available for depth calculation, but processing time increases due to large data volume

Engineering Contradiction:

Improvedepth determination accuracyVSAvoidprocessing time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent extracts only the essential information needed for depth calculation - specifically, the position of the illumination pattern image on the pixel array - rather than storing complete intensity values from all pixels. This is achieved by detecting the center position of the illumination pattern through moment calculations or correlation methods, thereby eliminating the need to store redundant intensity data while preserving the critical positional information required for accurate depth determination.

Inventive Principle:

Principle #2Taking out (Extraction)

3Measurement precision

If high pixel resolution is implemented in conventional systems, then depth map resolution improves, but readout time increases due to the large number of pixels

Engineering Contradiction:

Improvedepth map resolutionVSAvoidreadout speed

Core Design Contradiction:

Measurement precisionVSSpeed

Solution Approach 1:

The patent replaces the conventional approach of reading out and processing intensity values from all pixels with an event-driven system where only pixels detecting changes in illumination pattern position generate events. This substitution of the mechanical readout process with an event-based detection mechanism enables high-resolution depth mapping without the readout time penalty, as the system naturally filters and processes only relevant positional change information.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach reduces memory requirements, decreases latency, and enhances the accuracy of depth determination by focusing on event data, enabling higher pixel densities and faster readout times.

Implementation Method 1

a receiver unit comprising a plurality of pixels, the receiver unit being configured to detect on each pixel intensities of light reflected from the object

Methodology Applied

Scientific EffectPhotoelectric Effect: Photoelectric Effect

Data Source

PatentUS20250020455A1Depth sensor device and method for operating a depth sensor device

Publication Date: 2025.01.16 SONY SEMICON SOLUTIONS CORP

AI summary

A depth sensor device for measuring a depth map of an object comprises a projector unit configured to illuminate different locations of the object during different time periods with an illumination pattern, a receiver unit comprising a plurality of pixels, the receiver unit being configured to detect on each pixel intensities of light reflected from the object while it is illuminated with the illumination pattern, and to generate an event at one of the pixels if the intensity detected at the pixel changes by more than a predetermined threshold, and a control unit configured to generate for each of the different time periods a total number of detected events and pixel information indicating for each event the pixel that detected the event, and to calculate from the pixel information and the total number a position of the image of the illumination pattern on the pixels with sub-pixel accuracy.