Deep Learning View Switching for Real-Time Video Composition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video processing solutions fail to provide real-time view switching with improved compositional quality and efficiently use computing resources, often requiring multiple video files and cumbersome editing processes.
Innovation Solution
A deep learning system is integrated into a video processing application that renders a major view and suggests cropped videos with high compositional scores, allowing users to switch views seamlessly while performing all processing on the end-user device, reducing the need for multiple file versions and centralized computing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple video files are created and stored to support view switching, then view customization capability is improved, but computing resources (storage, processing power, memory) are wasted
Solution Approach 1:
Instead of creating and storing multiple actual video files for different views, the system creates virtual copies through software-based view switching. A single video file is stored, and multiple views are generated dynamically using image processing techniques such as cropping, zooming, and panning on the fly, eliminating the need to duplicate storage resources while maintaining full view customization capability
Solution Approach 2:
The video processing system is designed to perform multiple functions from a single video source. The same original video file serves as the basis for generating various views (full scene, zoomed-in views, panned views) through software processing, making the single video file universal for all viewing needs rather than requiring separate files for each view
2Manufacturing precision
If view switching is implemented without real-time processing, then compositional quality can be improved, but user experience is degraded due to delays
Solution Approach 1:
The system performs preliminary analysis of the video content to identify key objects, scenes, and compositional elements before view switching occurs. By pre-processing the video data to detect objects and determine optimal crop regions in advance, the system can rapidly switch between views with high compositional quality without introducing perceptible delays to the user experience
3Productivity
If deep learning system processes video in real-time on end-user device, then view switching speed is improved, but device computing resources are consumed
Solution Approach 1:
The deep learning system on the end-user device does not process the entire video stream for every view switching operation. Instead, it performs partial processing by analyzing only the current frame or key frames to identify objects and determine optimal crop regions. This selective processing maintains real-time view switching capability while significantly reducing power consumption compared to continuous full-video processing
Data Source
AI summary
Various embodiments describe view switching of video on a computing device. In an example, a video processing application executed on the computing device receives a stream of video data. The video processing application renders a major view on a display of the computing device. The major view presents a video from the stream of video data. The video processing application inputs the stream of video data to a deep learning system and receives back information that identifies a cropped video from the video based on a composition score of the cropped video, while the video is presented in the major view. The composition score is generated by the deep learning system. The video processing application renders a sub-view on a display of the device, the sub-view presenting the cropped video. The video processing application renders the cropped video in the major view based on a user interaction with the sub-view.