Image Sensor Frame Rate Control via Synthesized Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge is to maintain image quality while increasing the frame rate of an image sensor without unnecessary power consumption, as higher frame rates can lead to decreased exposure time and dark images, and independent operation of the image sensor and frame rate converter can result in image quality degradation and increased power usage.
Innovation Solution
An electronic device controls the frame rate of an image sensor by using a frame rate converter, which generates synthesized image frames to maintain exposure time and adjust the frame rate based on similarity thresholds, ensuring image quality and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the frame rate of the image sensor is increased to capture rapidly moving subjects, then the ability to capture moving subjects is improved, but the exposure time decreases causing images to look dark and quality to degrade
Solution Approach 1:
A frame rate converter is introduced as an intermediary component between the image sensor and the processing system. This converter generates intermediate image frames to increase the output frame rate without requiring the image sensor to operate at the higher frame rate, thereby maintaining adequate exposure time while achieving the desired high frame rate output
Solution Approach 2:
The system dynamically adjusts the frame rate of the image sensor based on motion detection and similarity comparison. When motion is detected between consecutive frames, the frame rate is increased; when no significant motion is detected, the frame rate is reduced. This parameter change allows the system to maintain high frame rate only when necessary, preserving image quality and brightness
2Illumination intensity
If the frame rate of the image sensor is increased to maintain exposure time, then the image quality is improved, but the power consumption increases
Solution Approach 1:
The image sensor operates dynamically with variable frame rate rather than a fixed high frame rate. The frame rate is adjusted in real-time based on detected motion and similarity thresholds, allowing the sensor to consume less power when high frame rates are not necessary while still maintaining image quality when motion is present
Solution Approach 2:
The frame rate converter acts as a mediator that decouples the image sensor operation from the required output frame rate. This allows the sensor to operate at lower, more power-efficient frame rates while the converter generates additional frames through interpolation and synthesis techniques
3Device complexity
If the image sensor and frame rate converter operate independently, then the device complexity is reduced, but the image quality is degraded due to differences between captured frames and generated intermediate frames
Solution Approach 1:
The system implements feedback mechanisms where the frame rate converter analyzes the similarity between consecutive image frames and adjusts its operation accordingly. When similarity exceeds a threshold indicating no significant motion, the converter reduces its activity. This feedback loop ensures that intermediate frames are generated only when necessary, maintaining image quality consistency while reducing unnecessary processing
Solution Approach 2:
The frame rate converter is integrated with the image sensor system, allowing coordinated operation through shared control logic and data flow. This merging enables the converter to receive raw sensor data and output processed frames in a unified pipeline, improving quality consistency while managing complexity through integrated design
4Speed
If the frame rate conversion is performed continuously to maintain high frame rate, then the frame rate is improved, but the power consumption increases unnecessarily when motion is absent
Solution Approach 1:
The frame rate conversion operates periodically rather than continuously, activated only when motion is detected between frames. The system monitors frame similarity and triggers frame rate conversion only when changes exceed a threshold, creating a periodic action pattern that reduces power consumption during static scenes while maintaining high frame rate when motion occurs
Solution Approach 2:
The frame rate of the image sensor is dynamically adjusted based on detected motion and similarity metrics. The sensor operates at high frame rate when motion is present and reduces to lower frame rate when scenes are static, creating a dynamic operation mode that optimizes power consumption while maintaining the ability to capture motion when it occurs
Data Source
Figure 1
Figure 2~3
Figure 4a
AI summary
An electronic device for controlling a frame rate and a method therefor are provided. The electronic device includes an image sensor, and a processor. The processor is configured to sequentially obtain a first image, a second image, and a third image through the image sensor at a first time interval, generate a synthesized image by using the first image and the third image, identify a similarity between the second image and the synthesized image, sequentially obtain a fourth image, a fifth image, and a sixth image through the image sensor, at a second time interval long than the first time interval, based on the similarity being equal to or greater than a first threshold value, and sequentially obtain the fourth image, the fifth image, and the sixth image through the image sensor, at the first time interval, based on the similarity being less than a first threshold value.