Synthesizing Intermediary Frames for Handheld Long Exposure Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Photographers face challenges in capturing sharp long exposure images without the use of tripods or additional equipment, as small involuntary muscle movements during multi-second exposures can cause camera motion, leading to unwanted blur in images taken with mobile devices.
Innovation Solution
A technique for creating long exposure images on mobile devices by blending multiple short exposures, using real-time image alignment and merging, and optionally incorporating synthetic frames to mitigate camera motion and enhance image stability, allowing for handheld photography without extraneous equipment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple short exposures are blended to create long exposure images, then image stability and reduction of camera motion blur are improved, but device complexity and processing requirements increase
Solution Approach 1:
The patent segments the long exposure capture process into multiple discrete short exposure frames, each captured independently. By dividing the total exposure time into multiple segments (e.g., 5-10 frames of 0.5-1 second each), the system can process and align individual frames to reduce the impact of camera motion, thereby improving image stability while managing processing complexity through incremental blending.
Solution Approach 2:
The patent applies preliminary alignment and registration actions to each captured frame before blending them together. By pre-processing each frame to correct for detected camera motion (using sensor data or feature matching) before the blending operation, the system improves final image stability without requiring excessively complex real-time processing during the merge operation.
2Stability of the object's composition
If synthetic intermediary frames are generated to fill gaps between exposures, then image smoothness and continuity are improved, but computational load and processing time increase
Solution Approach 1:
The patent introduces synthetic intermediary frames as mediator elements between the actual captured frames. These synthetic frames are generated by interpolating optical flow information or using motion estimation techniques to create intermediate images that fill temporal gaps. This approach smooths the transition between exposures and improves image continuity while allowing the system to work with fewer actual captured frames, potentially reducing total processing time.
Solution Approach 2:
The patent creates synthetic copies of frame content by warping and interpolating existing frame data. Instead of capturing additional real frames, the system generates synthetic copies through image processing operations (such as optical flow-based warping or neural network-based synthesis), achieving smooth transitions without the time cost of additional physical captures or complex processing.
3Manufacturing precision
If real-time alignment and merging of frames is performed, then image quality and motion compensation are improved, but energy consumption and device resource usage increase
Solution Approach 1:
The patent applies partial alignment and processing actions to only the necessary portions of each frame rather than processing entire high-resolution images. By focusing computational resources on key regions (such as areas with detected motion or features of interest) and using downscaled versions for alignment calculations, the system maintains high image quality where needed while significantly reducing overall energy consumption and device resource usage.
4Ease of operation
If handheld photography is enabled without tripods, then ease of operation and accessibility are improved, but image sharpness and stability deteriorate due to camera motion
Solution Approach 1:
The patent implements feedback loops that use data from device sensors (accelerometers, gyroscopes, magnetometers) to detect camera motion during the multi-frame capture sequence. This motion information is fed back into the frame alignment and blending process, allowing real-time compensation for handheld camera movements. The system continuously adjusts alignment parameters based on detected motion, enabling handheld operation while maintaining image sharpness through active motion correction.
Data Source
AI summary
Techniques for synthesizing intermediary frames for long exposure images are disclosed. Video stream data, comprising a plurality of arriving frames, is received. An indication that the video stream data should be processed into a long exposure image that incorporates one or more synthesized frames is received. A set of arriving frames is used to generate at least one synthesized frame. At least one received frame is blended with the at least one generated synthesized frame to form a long exposure image. The long exposure image is provided as output.