Depth Camera Image Stabilization for Video Jitter Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Handheld devices used for video recording often result in shaky footage due to user movement, leading to frame-to-frame jitter in video communications, which affects the quality of video content during video chats, conferences, and other video communications.
Innovation Solution
Implementing real-time depth camera-based image stabilization that collects depth data from a depth-aware camera on the source device to create stabilized video streams by identifying feature points, aligning frames, determining overlapping sections, and cropping or replacing frames to remove jitter, thereby stabilizing the video feed during communication sessions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If handheld devices are used for video recording, then ease of operation is improved, but video stability deteriorates due to user movement causing frame-to-frame jitter
Solution Approach 1:
The patent introduces depth information as an intermediary element that mediates between the handheld camera's unstable movements and the final stabilized video output. By using depth data from depth-aware cameras to identify foreground objects and compute motion vectors, the system creates a stable virtual camera view that compensates for physical hand movements, thus resolving the contradiction between ease of handheld operation and video stability
Solution Approach 2:
The patent replaces mechanical stabilization methods with computational image processing. Instead of physically stabilizing the camera through mechanical means, the system uses algorithms to process video frames, utilizing depth information to calculate motion vectors and reconstruct stable images, thereby substituting mechanical stabilization with computational approaches
2Stability of the object's composition
If depth camera-based image stabilization is implemented, then video stability is improved, but device complexity increases
Solution Approach 1:
The patent makes the depth-aware camera serve multiple functions: it not only captures depth information for stabilization but also identifies foreground objects, computes motion vectors, and provides data for virtual camera positioning. By making the depth camera multi-functional, the system reduces the need for separate stabilization hardware, thus managing device complexity while achieving video stability
Solution Approach 2:
The patent creates a virtual copy of the camera view that is independent from the physical camera's movements. By rendering a virtual camera view based on depth information and motion vectors, the system produces a stable video output without requiring the physical camera to be stabilized, thus managing complexity through computational rather than mechanical means
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A processing device collects depth data for frames in a sequence of images of a video stream being provided by a source device to a target device as part of a communication session. The depth data is created by a depth aware camera of the source device. The processing device maps, using the depth data, feature locations of the features of an object in a frame to feature locations of the features of the object in other frames, determines overlapping frame sections between the frames using the mapped feature locations, modifies, in the sequence of images, a set of images corresponding to the frames based on the overlapping frame sections to create a stabilized stream of images for the video stream, and provides the stabilized stream of images in the video stream as part of the communication session.