Same-Focal-Plane RGB-IR Pixel Layout for Aligned Image Sensing

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing image sensor technologies require separate modules for visible light and infrared (IR) image capture, leading to reduced display area, alignment issues, and decreased image quality due to the need for one sensor to pass IR light, which interferes with visible light capture and reduces signal-to-noise ratio.

Innovation Solution

A sensor module with an array of silicon-based visible light image sensor pixels and IR sensor pixels in the same focal plane, where IR sensor pixels are interspersed with visible light pixels, eliminating the need for IR light to pass through the visible light sensor and using thermally annealed silicon-based image sensor pixels with IR-sensitive material deposited in cavities defined by isolation walls.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If visible light and IR image sensors are integrated in the same plane with some pixels replaced by IR sensor pixels, then IR sensitivity is improved, but the quality of visible light image deteriorates and de-mosaicing complexity increases

Engineering Contradiction:
ImproveIR sensitivityVSAvoidvisible light image quality
Core Design Contradiction:
Measurement precisionVSManufacturing precision

Solution Approach 1:

The sensor array is segmented into distinct visible light sensor pixels and IR sensor pixels, with each type optimized for its specific function. The isolation walls create clear boundaries between pixels, enabling independent optimization of each sensor type without compromising the other's performance.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

Different regions of the sensor array have specialized properties: visible light pixels are optimized for visible spectrum detection while IR pixels are optimized for infrared detection. Each pixel type has tailored optical and electrical characteristics suited to its specific sensing function, improving overall system performance.

Inventive Principle:
Principle #3Local quality

2Manufacturing precision

If IR sensor pixels are introduced sparsely to maintain visible light image quality, then visible light image quality is preserved, but IR image quality becomes poor with limited applications

Engineering Contradiction:
Improvevisible light image qualityVSAvoidIR image quality
Core Design Contradiction:
Manufacturing precisionVSMeasurement precision

Solution Approach 1:

The patent merges visible light sensing and IR sensing capabilities into a single integrated sensor array. Both sensor types share the same focal plane and are processed together by the readout circuitry, enabling full-resolution imaging for both visible light and IR while maintaining image alignment.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The sensor array is designed with multi-functionality, where each pixel can be configured for either visible light or IR detection based on its intended use. The readout circuitry and processing system are universally capable of handling both sensor types, maximizing the utility of each pixel.

Inventive Principle:
Principle #6Universality (Multi-functionality)

3Measurement precision

If visible light and IR image sensors are stacked with IR sensor behind visible light sensor, then full resolution images are acquired and alignment is ensured, but IR light may be absorbed or reflected by visible light sensor components reducing SNR

Engineering Contradiction:
Improveimage resolution and alignmentVSAvoidIR light signal strength
Core Design Contradiction:
Measurement precisionVSLoss of energy

Solution Approach 1:

Instead of stacking the IR sensor behind the visible light sensor, the patent inverts the approach by placing both sensor types in the same focal plane. This eliminates the problem of IR light being absorbed or reflected by visible light sensor components, as IR light directly reaches the IR pixels without passing through visible light sensor layers.

Inventive Principle:
Principle #13The other way round (Inversion)

4Measurement precision

If visible light and IR image sensors are stacked in different focal planes, then full resolution images are acquired, but images are slightly out of focus requiring additional processing

Engineering Contradiction:
Improveimage resolutionVSAvoidimage focus alignment
Core Design Contradiction:
Measurement precisionVSManufacturing precision

Solution Approach 1:

The patent merges both visible light and IR sensors into the same focal plane, eliminating focus alignment issues. Both sensor types receive light from the same optical path and are focused at the same distance, ensuring that visible light and IR images are inherently aligned without requiring additional processing.

Inventive Principle:
Principle #5Merging (Combining)

5Reliability

If two separate camera modules are used for visible light and IR imaging, then each module is optimized for its specific function, but the area required increases limiting display area and making combined image use difficult

Engineering Contradiction:
Improvesensor optimizationVSAvoiddevice area
Core Design Contradiction:
ReliabilityVSArea of stationary object

Solution Approach 1:

The patent combines two separate camera modules into a single integrated sensor array. Both visible light and IR sensing functions are merged into one physical package with shared optics and processing, reducing the area required while maintaining the optimization benefits of having dedicated sensor regions for each function.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The integrated sensor array provides multi-functionality, serving both visible light imaging and IR imaging purposes within a single module. The readout circuitry and processing system are designed to handle both sensor types, enabling the device to perform multiple functions without requiring separate dedicated hardware for each.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This configuration enhances IR sensitivity, reduces optical crosstalk, and improves the modulation transfer function for both visible light and IR sensors, allowing for full resolution and aligned images while maintaining high quality.

Implementation Method 1

An IR-sensitive material in the array of cavities forms an array of IR sensor pixels in a same focal plane as the array of silicon-based image sensor pixels

Methodology Applied
Scientific EffectInfrared detection: Absorption (EM radiation)

Implementation Method 2

thermal annealing to integrate IR-sensitive material

Methodology Applied
Scientific EffectThermal annealing: Annealing

Data Source

PatentUS20240107782A1Same Focal Plane Pixel Design for RGB-IR Image Sensors
Publication Date: 2024.03.28 APPLE INC
  • US20240107782A1 patent drawing
  • US20240107782A1 patent drawing
  • US20240107782A1 patent drawing

AI summary

A sensor module includes a silicon substrate. A set of isolation walls defines, in the silicon substrate, an array of silicon-based image sensor pixels and an array of cavities. An infrared (IR)-sensitive material in the array of cavities forms an array of IR sensor pixels in a same focal plane as the array of silicon-based image sensor pixels.