Head-Mounted Image Processing with Dual-Shutter Flicker Repair
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Head-mounted devices, such as VR devices, suffer from stroboscopic phenomena in videos or images acquired by RGB cameras, leading to low display efficiency due to brightness flickering.
Innovation Solution
Utilize a first camera in rolling shutter mode and a second camera in global shutter mode to obtain images, perform image difference analysis to extract target gain information, repair the first image in the frequency domain using this information, and convert it back to the time domain to eliminate stroboscopic phenomena.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a rolling shutter mode is used to acquire images, then the device complexity is reduced, but stroboscopic phenomena occur causing display efficiency to deteriorate
Solution Approach 1:
The patent uses a global shutter camera as an intermediary device to capture reference images that are free from stroboscopic effects. These reference images serve as a mediator to provide stable frequency domain information that compensates for the rolling shutter images, allowing the system to maintain low device complexity while eliminating display efficiency degradation.
Solution Approach 2:
The patent transforms the image data from time domain to frequency domain through Fourier transformation, changing the parameter representation. By operating in the frequency domain and applying compensation based on global shutter reference images, the system resolves the stroboscopic issue without requiring hardware changes to the rolling shutter camera.
2Manufacturing precision
If multiple cameras are used to acquire images, then image quality is improved, but device complexity increases
Solution Approach 1:
The patent segments the imaging function into two specialized cameras: a rolling shutter camera optimized for color image acquisition and a global shutter camera optimized for capturing stroboscopic-free reference images. This segmentation allows each camera to perform its specific function efficiently while the processing system combines their outputs to achieve overall image quality improvement without excessive complexity.
Solution Approach 2:
The global shutter camera serves multiple functions: it captures reference images for frequency domain compensation, provides timing synchronization information, and enables the rolling shutter camera to maintain its color imaging capability. This multi-functionality justifies the addition of the second camera by maximizing its utility across different operational requirements.
3Object-affected harmful factors
If image processing in frequency domain is performed, then stroboscopic phenomena are eliminated, but processing time increases
Solution Approach 1:
The patent applies frequency domain processing selectively rather than to all images. Only the rolling shutter images requiring stroboscopic compensation undergo Fourier transformation and frequency domain compensation using reference images from the global shutter camera. Images from the global shutter camera skip this processing step, reducing overall processing time while still eliminating stroboscopic phenomena where needed.
Data Source
AI summary
Embodiments of the present disclosure provide an image processing method and apparatus, a device, a medium and a product. The method is applied to a head-mounted device, where the head-mounted device includes a first camera set to be in a rolling shutter mode and a second camera set to be in a global shutter mode, and the method includes: obtaining a first image acquired by the first camera and a second image acquired by the second camera; obtaining target gain information between the second image and the first image by performing image difference analysis on the first image and the second image; obtaining a target frequency domain image by using the target gain information to repair the first image in a frequency domain; obtaining a target image corresponding to the first image by converting the target frequency domain image from the frequency domain to a time domain.