ToF 3D Camera Malfunction Detection via Optical Image Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a lack of straightforward approaches for detecting random hardware faults in the light to electrical signal conversion of Time-of-Flight (ToF) based 3D-cameras and optical 2D cameras, which is crucial for ensuring functional safety in systems like LiDAR and automotive applications.
Innovation Solution
A method and apparatus that receive a luminance image from a ToF based 3D-camera and a 2D optical image from an optical 2D-camera, comparing them to determine if there is a malfunction by matching the luminance information, allowing for the detection of faults in the cameras' optics, pixels, and other elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional safety mechanisms are used to cover the electrical domain of ToF based systems, then electrical faults can be detected, but random hardware faults in the light to electrical signal conversion (pixel matrix, package, optics) cannot be detected
Solution Approach 1:
The patent introduces a second optical camera as an intermediary device to detect faults in the first ToF camera's optical path and pixel matrix. The second camera captures images that serve as reference data for comparing against images from the first camera, enabling indirect detection of hardware faults without requiring complex self-diagnostic capabilities in the faulty device.
Solution Approach 2:
The patent uses the second optical camera to create a copy or reference representation of the scene that can be compared with the first camera's images. This copying approach allows verification of the first camera's optical path and pixel matrix functionality by checking whether the reference image matches the captured image, thereby detecting hardware faults through comparison.
2Reliability
If no fault detection method is implemented for light to electrical signal conversion, then device complexity remains low, but functional safety cannot be ensured
Solution Approach 1:
The patent implements a feedback mechanism where images from the second optical camera are continuously compared with images from the first TOF camera to provide real-time feedback on the health status of the first camera's optical path and pixel matrix. This feedback loop enables ongoing monitoring and detection of hardware faults, ensuring functional safety through continuous verification.
Solution Approach 2:
The patent performs preliminary action by capturing reference images with the second camera before or during normal operation of the first camera. These pre-captured reference images serve as baseline data for detecting deviations that indicate hardware faults, allowing for proactive detection before safety-critical failures occur.
3Reliability
If images from TOF based 3D-camera and optical 2D-camera are compared to detect malfunction, then hardware faults can be detected, but processing time and computational resources increase
Solution Approach 1:
The patent extracts and compares only the most critical information from the images - specifically the luminance or intensity data that corresponds to the optical path and pixel matrix functionality. By focusing the comparison on this essential data rather than processing entire complex images, the system reduces computational overhead and processing time while still effectively detecting hardware faults.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables continuous checking of the optical to electrical conversion in 3D and 2D cameras, effectively detecting hardware faults and ensuring the reliability of the sensor data, thereby enhancing functional safety in applications like autonomous vehicles.
Implementation Method 1
In ToF based systems like Light Detection And Ranging (LiDAR) or 3D imaging, light is converted into an electrical signal
Data Source
AI summary
A method for determining malfunction is provided. The method includes receiving a 1D or 2D luminance image of a scene from a time-of-flight based 3D-camera. The luminance image includes one or more pixels representing intensities of background light received by an image sensor of the 3D-camera. The method further includes receiving a 2D optical image of the scene from an optical 2D-camera and comparing the luminance image to the optical image. If the luminance image does not match the optical image, the method additionally includes determining malfunction of one of the 3D-camera and the 2D-camera.