Light Receiving Pixel Filters for Optical Convolution Imaging

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Image recognition processing by convolution operations faces challenges in achieving high real-time performance due to the large amount of data to be processed and the complexity of the processing itself.

Innovation Solution

A light receiving device comprising first filters that transmit edge components in predetermined directions, second filters that transmit specific wavelength bands, and photoelectric conversion elements that convert light through these filters, enabling optical convolution operations for faster image recognition.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If image recognition processing is performed by convolution operation on image data, then recognition accuracy is improved, but processing speed deteriorates due to large data amount and complex processing

Engineering Contradiction:
Improverecognition accuracyVSAvoidprocessing speed
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The patent replaces electronic/digital convolution operations with optical convolution operations. By using optical elements (lenses, mirrors, spatial light modulators) to perform the convolution mathematically in the optical domain, the system achieves parallel processing of all pixel data simultaneously, thereby maintaining high recognition accuracy while dramatically improving processing speed.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent transforms the convolution operation from the spatial domain to the frequency domain using optical Fourier transforms. This dimensional transformation allows the convolution operation to be performed as a simple multiplication in the frequency domain, reducing computational complexity from O(N²) to O(N) while preserving recognition accuracy.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Productivity

If optical convolution operations are performed using filters for edge components and wavelength bands, then processing speed is improved, but device complexity increases

Engineering Contradiction:
Improveprocessing speedVSAvoiddevice complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent designs optical elements that can perform multiple functions: the same optical convolution system can be reconfigured to perform different convolution operations by changing the spatial light modulator patterns or filter configurations. This multi-functionality reduces the need for multiple dedicated hardware components, thereby improving processing speed without proportionally increasing device complexity.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent employs programmable spatial light modulators and tunable filters that can dynamically change their characteristics during operation. This dynamic reconfigurability allows the system to adapt to different recognition tasks without requiring physical reconfiguration of the entire optical system, balancing processing speed improvement with manageable device complexity.

Inventive Principle:
Principle #15Dynamics

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This configuration allows for higher-speed image recognition processing by performing convolution operations optically, potentially omitting the need for the first convolution layer in neural networks and enhancing processing speed and accuracy.

Implementation Method 1

a plurality of photoelectric conversion elements that each photoelectrically convert light transmitted through one of the plurality of first filters and one of the plurality of second filters

Methodology Applied
Scientific EffectPhotoelectric conversion: Photoelectric Effect

Data Source

PatentUS11928848B2Light receiving device, solid-state imaging apparatus, electronic equipment, and information processing system
Publication Date: 2024.03.12 SONY SEMICON SOLUTIONS CORP
  • US11928848B2 patent drawing
  • US11928848B2 patent drawing
  • US11928848B2 patent drawing

AI summary

Higher-speed image recognition processing can be implemented. A light receiving device according to an embodiment includes: a plurality of first filters (130) each transmitting an edge component in a predetermined direction in an incident image; a plurality of second filters (150) each transmitting light of a predetermined wavelength band in incident light; and a plurality of photoelectric conversion elements (PD) each photoelectrically converting light transmitted through one of the plurality of convolution filters and one of the plurality of color filters.