Camera Sensor HDR Stream and Exposure Data Exchange
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In camera systems, especially in automotive applications, it is challenging to obtain individual exposure-specific statistics for camera sensor calibration when high-dynamic range (HDR) combining is performed by on-sensor processing systems, as exposure-level information is lost during the merging process, and extracting this information increases the complexity and cost of the camera sensors.
Innovation Solution
A method where a camera sensor module captures multiple exposures, performs HDR combining, and communicates both the HDR-combined stream and individual exposures to a vision processing system, allowing for color processing to collect exposure-specific statistics for camera calibration, thereby preserving exposure-level information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If HDR-combining is performed by on-sensor processing systems integrated with camera sensors, then HDR image stream quality is improved, but exposure-level information is lost making calibration difficult
Solution Approach 1:
The patent extracts exposure-level information from the HDR-combined stream by using a second image signal processor to analyze the original individual exposures that were used to create the HDR image. This allows calibration statistics to be collected without degrading the HDR image quality produced by the on-sensor processing system.
Solution Approach 2:
The vision processing system is designed with multiple image signal processors that serve different functions: one ISP processes the HDR-combined stream for final image output, while another ISP processes the individual exposures to extract calibration statistics. This multi-functionality allows the system to simultaneously achieve high HDR quality and maintain access to exposure-level information for calibration.
2Measurement precision
If exposure-level information is extracted by integrated on-sensor processing systems, then calibration capability is improved, but device complexity and cost increase
Solution Approach 1:
The patent introduces an intermediary approach where the camera sensor module captures and outputs individual exposures, and a separate vision processing system with multiple ISPs performs the calibration analysis. This mediator architecture allows calibration functionality to be added without increasing the complexity of the camera sensor itself, as the calibration processing is performed externally on the captured exposure data.
Solution Approach 2:
Instead of integrating complex calibration processing directly into the camera sensor, the system creates a copy of the individual exposure data and processes it separately in the vision processing system. This allows the original camera sensor to remain simple while still enabling exposure-level analysis for calibration purposes through the copied data stream.
3Illumination intensity
If multiple exposures are merged to form HDR-combined stream, then dynamic range is improved, but individual exposure statistics are lost
Solution Approach 1:
The patent segments the processing of HDR data into two separate pathways: one pathway processes the merged HDR-combined stream for final image output, while another pathway processes the individual unmerged exposures to extract exposure-specific statistics. This segmentation allows both high dynamic range imaging and calibration statistics collection to occur simultaneously without interfering with each other.
Solution Approach 2:
The system performs preliminary action by capturing and preserving the individual exposures before they are merged into the HDR-combined stream. By maintaining these original exposure files, the system ensures that exposure-level information remains available for later calibration analysis, even after the HDR merging process has enhanced the dynamic range.
Data Source
AI summary
In an embodiment, a camera sensor module is communicatively coupled to a vision processing system. The camera sensor module captures a plurality of exposures, produces an HDR-combined stream, and sends both the HDR-combined stream and the plurality of exposures to the vision processing system. A first ISP of the vision processing system performs color processing to produce a color-processed HDR-combined stream for neural network processing, presentation to a human, and/or computer vision processing. At least one other ISP of the vision processing system performs color processing to collect exposure-specific statistics on at least one of the plurality of exposures for determining camera calibration data for the camera sensor module.