Inertial Sensor-Based Exposure Smoothing for Video Stability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional auto white balance (AWB) and auto exposure (AE) algorithms in cameras often result in undesirable tonal fluctuations and exposure variations, especially in mobile camera video sequences, due to frequent recalculations based on per-frame analysis.
Innovation Solution
The implementation of inertial sensors, such as accelerometers and gyroscopes, to adjust the frequency of AWB and AE calculations, storing recent frame settings in a memory buffer for interpolation, thereby reducing fluctuations by recalculating settings only when the camera moves a predetermined distance or experiences significant illumination changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If per-frame analysis is used for AWB and AE calculations, then responsiveness to lighting changes is improved, but tonal fluctuations and exposure variations increase
Solution Approach 1:
The patent implements a dynamic frame selection mechanism that adapts the frequency of AWB and AE calculations based on detected camera motion. When motion is detected via inertial sensors, the system increases calculation frequency to maintain responsiveness; when motion is minimal, it reduces frequency to prevent fluctuations. This dynamic adjustment resolves the contradiction between responsiveness and tonal stability.
Solution Approach 2:
The patent introduces inertial sensors as an intermediary between the camera and the AWB/AE algorithms. These sensors detect camera motion and provide feedback that mediates the calculation frequency, preventing unnecessary recalculations during stable periods while maintaining responsiveness during motion. This intermediary layer decouples the direct per-frame analysis that causes fluctuations from the lighting adaptation requirement.
2Adaptability or versatility
If AWB and AE calculations are performed frequently, then adaptability to changing conditions is improved, but computational overhead increases
Solution Approach 1:
The patent implements periodic AWB and AE calculations triggered by motion events rather than continuous per-frame analysis. The inertial sensors detect motion periods, and calculations are performed only during these periods. This periodic approach maintains adaptability to actual lighting changes while significantly reducing computational overhead during stable periods when no motion occurs.
Solution Approach 2:
The system uses the camera's own motion detection capabilities (via inertial sensors) to self-regulate the calculation frequency. The camera monitors its own movement and automatically adjusts the AWB/AE calculation rate accordingly, eliminating the need for external control and reducing unnecessary computational operations while maintaining adaptability when needed.
3Speed
If motion detection threshold is lowered, then responsiveness to camera movement is improved, but false triggering increases
Solution Approach 1:
The patent combines multiple motion detection parameters (acceleration magnitude, velocity changes, and temporal patterns) into a unified motion detection decision. Rather than relying on a single threshold, the system merges multiple sensor inputs and analysis dimensions to determine whether motion warrants AWB/AE recalculation. This merging approach improves responsiveness to genuine motion while filtering out false triggers through cross-validation of multiple parameters.
Solution Approach 2:
The system implements feedback mechanisms where previous motion detection results and calculation outcomes inform future threshold adjustments. When false triggering occurs, the system learns from this feedback and adjusts its detection parameters. This feedback loop maintains responsiveness to legitimate motion while progressively reducing false triggering through adaptive threshold management based on observed performance.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A first image frame is gathered. The first image frame, a first white balance value, and a first exposure value are stored. Information is sensed from an inertial sensor. A second image frameand intermediate image frames are gathered. A distance traversed by a camera is determined. In response to one or more of the distance exceeding a predetermined threshold, or a new white balance value significantly differing from a previous white balance value, or a new exposure value significantly differing from a previous exposure value, a second white balance value and a second exposure value are determined for the second image frame. A smoothed white balance value and a smoothed exposure value are determined for the intermediate image frames.