CW-TOF Camera Dual-Pixel Sensor for Range and Picture Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional time-of-flight (TOF) cameras can only provide range images using infrared light, lacking the capability to capture high-resolution visible light images simultaneously, which limits their ability to offer a comprehensive visual representation of a scene.
Innovation Solution
Incorporating both infrared (IR) pixels and visible light-sensitive 'picture' pixels in a TOF camera's photosensor, allowing a processor to estimate visible light intensities and generate a full-resolution picture image that matches the spatial configuration of the pixels, thereby providing both range and picture images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the photosensor uses only IR pixels with IR bandpass filters, then the camera can provide accurate range images using infrared light, but it cannot capture visible light images simultaneously
Solution Approach 1:
The patent combines IR pixels and visible light-sensitive picture pixels into a single photosensor array, allowing simultaneous capture of both infrared and visible light. This merging approach enables the camera to acquire both range image data and picture image data through one integrated sensor rather than requiring separate sensors, thereby improving versatility while managing device complexity.
Solution Approach 2:
The photosensor is designed with dual functionality: IR pixels for range measurement and picture pixels for visible light imaging. Each pixel type is optimized for its specific function, with IR pixels equipped with IR bandpass filters and picture pixels with visible light filters. This multi-functional design allows the camera to perform both TOF range imaging and conventional picture imaging using a single device.
2Productivity
If the camera uses a single photosensor for both IR and visible light, then it can provide both range and picture images, but the image resolution may be compromised due to pixel interspersion
Solution Approach 1:
The photosensor is segmented into distinct IR pixels and picture pixels, with each segment optimized for its specific function. The picture pixels are strategically interspersed among IR pixels at known locations, allowing the system to process and reconstruct full-resolution picture images by utilizing only the picture pixel data while maintaining the overall pixel array structure for range measurement.
Solution Approach 2:
Different regions of the photosensor have different functional qualities: IR pixels are optimized for infrared detection with appropriate filters, while picture pixels are optimized for visible light detection. This local differentiation allows each pixel type to perform its specific function at high quality, and the known spatial relationship between pixel types enables software reconstruction of full-resolution images.
3Reliability
If IR pixels are shielded from visible light using IR bandpass filters, then they can accurately detect infrared light for range measurement, but they cannot contribute to visible light picture images
Solution Approach 1:
The patent extracts the visible light detection function from the IR pixel locations by introducing dedicated picture pixels with visible light filters at those positions. The IR bandpass filters on IR pixels ensure they only detect infrared light for accurate range measurement, while the picture pixels at interspersed locations capture visible light information that would otherwise be lost, allowing reconstruction of complete picture images.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables the TOF camera to produce high-resolution color or monochrome picture images alongside range images, enhancing visual representation and image resolution without compromising the spatial homogeneity of the pixel configuration.
Implementation Method 1
the camera processes reflected light from the features that pixels in the photosensor register to provide a range image of the scene comprising measures of distances to the features. In a time of flight (TOF) range camera the camera processes reflected light from a feature that a pixel registers to determine a round trip flight time, 'τ R ', of light from the camera to the feature and back to the camera and therefrom a distance 'd' to the feature.
Implementation Method 2
the pixels, hereinafter also referred to as 'IR pixels', in the photosensor that provide distances to features in the scene are shielded from visible light by IR bandpass filters.
Implementation Method 3
pixels, also referred to as 'picture pixels', that are sensitive to visible light and are interspersed on the photosensor at known locations relative to the IR pixels
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A CW-TOF camera 20 operates to acquire a range image of a scene 130, optionally including objects 131 and 132, and provides distances to features in the scene and a picture image of the scene. CW-TOF camera 20 comprises an optical system represented by a lens 21, a light source 30 controllable to transmit, optionally IR, light to illuminate scene 130 and optionally a visible light source 30-V, which may be a white light source, controllable to illuminate scene 130 with visible, optionally white, light 31-V. CW-TOF camera 20 also comprises a photosensor 40 on which optical system 21 images IR light reflected by features in scene 130 from the transmitted IR light and visible light from visible light source 30-V and ambient light that the features reflect. The CW-TOF camera comprises an oscillator 50 that provides a modulation frequency for modulating IR light that light source 30 transmits, a phase shifter 52 for providing sampling phase offsets for sampling IR light reflected by features in scene 130 back to the camera., and a controller 54 that controls components comprised in the CW-TOF camera. Photosensor 40, a portion of which is shown in an inset 60, provides CW-TOF camera 20 with data for producing a range image of scene 130 and a color picture image of the scene. The photosensor comprises rows and columns of light sensitive pixels referenced generically as pixels 41. Pixels 41 comprise IR sensitive pixels 41-IR for registering reflected IR light from features in scene 130 and color pixels 41-R, 41-G, and 41-B for registering respectively R, G, and B light from the features. A processor in the TOF camera processes amounts of IR light that the IR pixels in the photosensor register and visible light from the scene that the color pixels in the photosensor register to estimate intensities of visible light incident on the IR and color pixels. The processor optionally uses the determined estimates to provide a picture image of the scene for which a visual light intensity is associated with each IR pixel as well as color pixel located in a region of the photosensor used to generate the picture image. The picture image therefore benefits from an image resolution that substantially matches that provided by the pitch of pixels in the photosensor, agnostic as to whether they are IR or color pixels.