High-Rate Video Generation for Motion Cameras
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current digital night vision cameras (NVCs) used in conjunction with augmented reality displays for dismounted soldiers face challenges in reducing latency and maintaining low light sensitivity, leading to user nausea and difficulty interpreting complex surroundings.
Innovation Solution
A system and method that generates a high-rate video for a camera in motion by using low-rate video captured by NVCs and high-rate inertial information to estimate camera motion, applying a geometrical warping algorithm to interpolate images and produce a high-rate video that counteracts head rotation and translation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the NVC frame rate is increased to 100-120 Hz to reduce latency and improve smoothness, then the video smoothness and latency are improved, but the low light sensitivity deteriorates due to reduced exposure time
Solution Approach 1:
The patent segments the video generation process into multiple components: capturing frames at low rate with high sensitivity, separately estimating camera motion at high rate using inertial sensors, and then synthesizing intermediate frames through geometric warping. This allows the system to achieve high output frame rate without requiring the sensor to operate at high frame rates, thus preserving low light sensitivity while improving video smoothness
Solution Approach 2:
The system performs preliminary action by capturing video frames at the moment of actual sensor exposure (when light sensitivity is maximized) and pre-estimating camera motion using inertial sensors between frames. This preliminary capture and motion estimation enable the synthesis of high-rate video without requiring the sensor to continuously expose at high rates, thereby maintaining low light sensitivity while achieving smooth playback
2Illumination intensity
If the NVC frame rate is kept low at 15-30 Hz to maintain low light sensitivity, then the low light sensitivity is preserved, but the video smoothness and latency worsen causing user nausea
Solution Approach 1:
The patent introduces inertial sensors as an intermediary that captures camera motion at high rates between video frames. This intermediary high-rate motion data serves as a bridge, allowing the system to generate smooth high-rate video output from low-rate sensor input. The inertial sensors mediate between the low frame rate capture and high frame rate display requirements, enabling smooth video without compromising low light sensitivity
Solution Approach 2:
The system changes parameters by operating the sensor at low frame rate for optimal light sensitivity, while simultaneously using inertial sensor data to estimate and compensate for camera motion at high rates. The geometric warping process dynamically adjusts image parameters based on estimated motion, effectively transforming low-rate captures into high-rate smooth video output without changing the sensor's operating frame rate
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method, medium and system for generating a high-rate video for displaying to a user on a display device. The method, medium and system may generate and display the high-rate video by: capturing images for a video at a first frame rate, capturing movement of a display device via a motion sensor, estimating a motion of the camera based on images of the video captured at the first frame rate and the movement of the display device captured via the motion sensor, generating the high-rate video at a second frame rate by up-sampling the video captured at the first frame rate, and displaying the high-rate video to the user.