CMOS Image Sensor ROI Interpolation for Low-Power Video Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hybrid EVS-CIS image sensors face high power consumption and long calculation times due to the conversion of EVS information to CIS information using optical flow, limiting their suitability for real-time video capture.
Innovation Solution
Implementing Event Guided Low Power (ELP) CIS VFI, which reduces power requirements and calculation time by reading pixel values only when event signals are detected, interpolating between key frames, and selectively processing regions of interest.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If optical flow conversion is used to convert EVS information to CIS information, then video frame interpolation can be achieved, but power consumption increases and calculation time increases
Solution Approach 1:
The patent extracts only the essential event data from EVS pixels that indicate luminance changes, rather than converting all EVS information through computationally intensive optical flow algorithms. This selective extraction of critical motion information reduces power consumption while maintaining interpolation accuracy.
Solution Approach 2:
The patent segments the image sensor operation into event-driven regions where luminance changes occur, processing only these segments rather than the entire frame. This segmentation allows video frame interpolation to be performed selectively, reducing overall calculation time and power consumption.
2Measurement precision
If optical flow conversion is used to convert EVS information to CIS information, then video frame interpolation can be achieved, but calculation time increases
Solution Approach 1:
The patent extracts only the essential event data from EVS pixels that indicate luminance changes, rather than converting all EVS information through computationally intensive optical flow algorithms. This selective extraction of critical motion information reduces power consumption while maintaining interpolation accuracy.
Solution Approach 2:
The patent performs preliminary filtering of event data at the pixel level, pre-identifying regions with luminance changes before interpolation processing. This preliminary action reduces the data volume requiring subsequent interpolation calculations, thereby reducing overall calculation time.
3Reliability
If all pixel values are read from the image sensor core, then complete video frames can be generated, but power consumption increases
Solution Approach 1:
The patent extracts only the essential event data from EVS pixels that indicate luminance changes, rather than converting all EVS information through computationally intensive optical flow algorithms. This selective extraction of critical motion information reduces power consumption while maintaining interpolation accuracy.
Solution Approach 2:
The patent applies different processing quality to different regions: event-driven regions receive full interpolation processing to maintain reliability, while static regions use simplified processing or are omitted entirely. This local quality differentiation reduces overall power consumption while maintaining video frame completeness where needed.
Data Source
AI summary
A CMOS image sensor generates a key frame by reading pixel values of all pixels from an image sensor core and outputs the pixel values as a key frame. The sensor generates an interpolated frame by reading at least one pixel value from a pixel area of the image sensor core having at least one pixel identified by an event signal, interpolating between a corresponding portion of the key frame and the at least one pixel value to form an interpolated partial frame; and outputting the interpolated partial frame. The CMOS image sensor determines a region of interest (ROI) from the event signal and interleaves precharge of full frame rows of the image sensor core with precharge of ROI partial rows of the ROI; and interleaves readout of full frame pixel values from the full frame rows with readout of ROI pixel values from the ROI partial rows.